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1  EXECUTIVE  SUMMARY 

Defence  R&D  Canada's  Centre  for  Security  Science  (CSS)  is  working  in  partnership  with  Public  Safety 
Canada  (PS)  on  a  long  term  objective  to  develop  an  integrated  national  and  regional  Critical 
Infrastructure  (Cl)  Dependency  model  for  Cl  risk  analysis  and  risk  mitigation  in  support  of  the  2010 
National  Critical  Infrastructure  Strategy.1  To  frame  discussions  and  serve  as  a  starting  point,  CSS  has 
requested  NRC  Knowledge  Management  to  perform  an  exploratory  study  of  existing  scientific,  industrial 
and  government  (domestic/international)  literature  on  critical  infrastructure  protection  and  resilience 
(CIPR)  and  related  concepts;  including,  but  not  limited  to,  Cl  interdependencies  as  well  as  modeling  and 
simulation  tools. 

This  report  focuses  on  modeling  and  simulation  tools  and  trends.  In  all,  2,151  publications,  including  a 
sub  dataset  of  633  publications  specifically  focused  on  interdependency  modeling  and  simulation,  were 
retrieved  and  analyzed  using  text  mining  software  and  a  variety  of  visualization  tools  to  identify  trends 
in  the  literature  and  key  players.  Additionally  a  listing  of  113  tools  found  in  the  dataset  are  also 
provided,  among  which  82  tools  focused  on  interdependency  modeling  or  simulation,  18  focused  on 
regional  or  local  modeling  or  simulation,  and  13  focused  on  government/business  continuity  modeling 
and  simulation  with  some  repeats  between  groups.  Government/business  continuity  tools  are  used 
most  frequently  to  model  and  simulate  CIPR  in  the  Energy  and  Utilities  and  Water  sectors.  Other 
highlights  of  the  datasets  are  provided  in  Table  1. 

The  similarities  between  the  two  data  sets  (CIPR  modeling  and  simulation  and  CIPR  interdependency 
modeling  and  simulation)  are  quite  notable.  It  is  possible  that  the  subfield  of  CIPR  interdependency 
modeling  and  simulation  is  still  depending  on  tools  and  techniques  used  in  general  CIPR  modeling 
because  it  has  not  yet  developed  enough  of  its  own  tools  and  techniques.  Ouyang2  argues  that  there  is 
still  a  need  for  a  new  concept  that  can  "integrate  different  approaches  into  a  single  framework  and  co¬ 
simulation  platform  to  address  different  aspects  of  interdependent"  critical  infrastructure.  This  view  is 
supported  by  the  general  lack  of  specific  best  practices  or  lessons  learned  in  the  literature  that  was 
reviewed,  i.e.  the  field  of  modeling  and  simulating  critical  infrastructures,  and  in  particular  Cl 
interdependencies,  has  not  yet  matured.  In  general,  articles  that  did  provide  best  practices,  lessons 
learned,  or  recommendations  were  either  presented  at  a  higher  level  of  managing  risks  or  disaster 
response  or  were  more  specific  to  the  tool  being  presented  and  not  easily  generalized  to  modeling  and 
simulation  of  Cl  interdependencies. 

Alternatively  it  may  be  that  the  tools  and  techniques  that  are  used  in  general  CIPR  modeling  are 
sufficiently  applicable  to  interdependency  modeling.  More  time  and  testing  of  these  techniques  would 
be  required  to  determine  the  true  cause  of  the  overlap.  Revisiting  the  numerous  projects  and  programs 
that  are  presented  would  allow  one  to  keep  a  finger  on  the  pulse  of  developments  in  this  area. 

A  number  of  agreed  upon  challenges  in  the  field  of  Cl  interdependency  modeling  and  simulation  include 
insufficient  access  to  data;  incorporating  less  commonly  studied  Cl  in  models  and  simulations  to  improve 
disaster  mitigation  and  recovery;  integrating  a  variety  of  tools  and  techniques  into  an  open  framework 
to  compensate  for  varying  levels  of  error,  uncertainty  and  conflicting  results;  validating  new  models  in 
ever  changing  environments  and  finally,  dealing  with  the  general  lack  of  theories  and  generic  results 
that  are  easily  generalized  to  actual  infrastructure. 

A  brief  overview  of  findings  pursuant  to  the  key  questions  of  the  project  is  included  in  Table  1. 
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Table  1.  Summary  of  Findings:  Key  Questions 

Topic 

CIPR  M&S  Tools  in  General 

CIPR  Interdependency  M&S  Tools 

Top  Author  Affiliations 

The  United  States,  Italy  and  Canada 
are  the  leading  countries  in  CIPR 
modeling  and  simulation,  with 
roughly  an  equal  distribution 
between  governmental  and 
academic  affiliations.  University  of 
British  Columbia,  York  University 
and  Western  University  top  the 
Canadian  affiliations  list. 

Polytechnic  University  of  Milan  (Italy), 
Huazhong  University  of  Science  and 
Technology  (China),  University  Campus 
Bio-Medico  of  Rome,  Rice  University 
(U.S.),  the  Italian  National  Research 
Council  (CNR),  TNO  Defense 
(Netherlands)  and  Gjpvik  University 
College  in  Norway.  University  of  British 
Columbia,  York  University  and  Western 
University  remain  the  top  the  Canadian 
affiliations 

Top  Critical  Infrastructures 

Energy  and  Utilities,  Information 
and  Communication  Technology, 
Safety,  Transportation  and  Cyber.3 

Energy  and  Utilities,  ICT, 

Transportation,  Safety  and  Water. 

Top  Threats 

Terrorism,  Disasters,  Earthquakes, 
CBRNE,  Natural  disasters 

Disasters,  Terrorism,  Earthquakes, 

Natural  disasters  and  Floods 

Common  M&S  Techniques 

Dynamic  Modeling  and  Simulation 
(M&S),  Estimation, 
Statistical/Numerical  techniques, 
Classification/Pattern  Identification, 
and  Geographic  Information 

Systems  (GIS)  and  Graph 
Theory/Models 

Dynamic  M&S,  Estimation,  GIS,  Input- 
Output  Modeling  and 
Classification/Pattern  Identification. 
Regional  tools  use  Dynamic  M&S,  DIIM, 
GIS  and  Forecasting.  Regional  tools  are 
used  for  M&S  Safety  critical 
infrastructures  as  much  as  Energy  and 
Utilities. 

Emerging  M&S  Techniques 

Behavioral  Analysis,  Monte  Carlo 
techniques,  Clustering  and  Petri 

Nets 

Bayesian  techniques,  Hierarchical 
Methods,  Clustering,  Graph 
Theory/Model,  Monte  Carlo,  Tree 

Analysis  and  Dynamic  Inoperability 
Input-Output  Model  (DIIM) 

New  or  Pre-Emerging 
Techniques 

Tree  Analysis,  Bayesian,  Input- 
Output  Modeling,  Fuzzy  techniques 
and  Hierarchical  Methods 

Inconclusive 

Outliers 

Game  Theory  for  terrorist  threats 

Input-Output  Modeling  used  equally  as 
frequently  for  Finance  as  for  Energy 
and  Utilities;  Hierarchical  methods  in 

ICT 

The  authors  would  like  to  thank  Tamara  Keating  at  the  National  Research  Council  of  Canada's  Knowledge 
Management  Department  for  her  helpful  review  and  comments  on  this  report. 


Cyber  was  considered  separately  from  ICT  because  of  the  volume  of  publications  that  focused  specifically  on  cyber  issues  and 
the  fact  that  less  than  half  co-occurred  with  ICT  thus  arguably  warranting  its  own  examination. 
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2  BACKGROUND 

2.1  Context 

In  2009,  the  Canadian  federal  government,  provinces  and  territories  agreed  on  a  National  Strategy  and 
Action  Plan  for  Critical  Infrastructure.  The  purpose  of  this  initiative  is  to  strengthen  the  resilience  of 
Canadian  Critical  Infrastructure  (Cl)  by  building  partnerships,  implementing  hazards  risk  management 
approaches,  and  advancing  the  timely  sharing  and  protection  of  information  among  partners.  The 
strategy  recognized  that  critical  infrastructures  are  at  risk  from  natural,  intentional  and  accidental 
hazards  and  that  the  risk  could  be  exacerbated  by  the  complex  system  of  interdependencies  among 
critical  infrastructure,  which  can  lead  to  cascading  effects  across  borders  and  sectors.  The  Action  Plan 
includes  the  establishment  of  sector  networks  and  a  cross-sector  forum  as  the  basis  for  collaborative 
work  and  information  sharing.  The  Centre  for  Security  Science  (CSS)  is  working  in  partnership  with 
Public  Safety  Canada  (PS)  on  initiatives  to  address  some  of  the  objectives  identified  within  the  strategy 
and  its  action  plan  in  order  to  increase  Canadian  infrastructure  resiliency,  develop  strong  communities 
and  implement  an  all-hazards  risk  management  approach.  The  continuity  of  national  governance  to 
maintain  public  health,  safety,  security,  economic  well-being  and  the  confidence  during  or  after  any 
disaster  or  emergency  is  one  of  the  initiatives.  The  development  of  national  and  regional 
interdependency  modeling  tools  and  methodologies  to  understand  Cl  interdependencies  and  the 
cascading  effect  of  events  is  another  important  initiative. 


2.2  Key  Issues 

To  frame  discussions  and  serve  as  a  starting  point  in  the  development  of  a  national  and  regional  Cl 
interdependency  modelling  tool,  CSS  has  requested  an  exploratory  study  of  existing  scientific,  industrial 
and  government  (domestic/international)  literature  on  critical  infrastructure  protection  and  resilience 
(CIPR)  and  related  concepts;  including,  but  not  limited  to,  Cl  interdependencies  as  well  as  modeling  and 
simulation.  This  study  will  examine  the  existing  body  of  knowledge  and  attempt  to  structure  the  current 
state  of  knowledge  of  CIPR  modeling  and  simulation  and  CIPR  interdependency  modeling  and 
simulation.  It  is  anticipated  that  the  results  of  this  work  will  yield  a  more  in  depth  and  enhanced 
understanding  of  the  concepts,  activities  and  tools  associated  with  CIPR  in  terms  of  physical  and  cyber 
security  systems  from  both  civilian  and  military  perspectives.  Additionally,  it  will  help  CSS  develop 
advanced  capabilities  and  expertise  in  the  area,  as  well  as  highlight  gaps,  lessons  learned  and 
opportunities  for  next  steps  in  the  pursuit  of  an  integrated  CIPR  strategy. 


2.3  Key  Questions 

1.  What  is  the  state  of  CIPR  modeling,  simulation  and  analysis  (both  in  terms  of  risk  modeling  for  Cl 
and  modeling  protection  and  resilience)?  Provide  an  overview  of  the  available  tools  (by  types, 
not  specific  names  of  tools). 

2.  Identify  and  provide  an  overview  of  tools  that  model,  simulate  and  analyze  CIPR 
interdependency  as  well  as  regional  (in  Canada  and  the  US)  CIPR  risks  and  capabilities.  Identify, 
when  possible,  the  applicability  and  challenges  for  these  tools. 

3.  Are  there  any  tools  that  focus  on  Cl  interdependency  modeling  in  support  of  government 
continuity? 

4.  What  are  the  key  observations/lessons  learned  in  Cl  modeling  particularly  interdependency 
modeling? 
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3  INTRODUCTION 


To  address  the  key  questions,  data  was  gathered  from  four  bibliographic  databases.  The  complete  list  of 
sources  and  the  search  strategy  are  described  in  detail  in  the  Methodology  section  (7)  of  this  report.  The 
dataset  was  divided  into  two  subsets.  The  first,  referred  to  hereafter  as  the  master  set,  includes  all 
records  found  in  the  search  with  a  total  of  2151  records  published  between  2003  and  2013.  The  second, 
referred  to  hereafter  as  the  interdependency  set,  was  a  sub  dataset  created  by  copying  all  records  that 
had  the  term  interdependency,  independency,  dependent,  dependency,  regional,  or  local  (and  any 
other  synonyms  in  the  dataset)  and  totalled  633  records.  Analyses  on  the  master  set  include  records 
from  the  interdependency  set. 

Figure  1  depicts  the  publication  rate  for  both  datasets. b  While  both  lines  follow  a  similar  pattern,  and 
both  sets  show  a  slight  drop  in  2012,  we  notice  a  much  steeper  climb  in  the  master  set  since  2008  versus 
a  flattening  of  the  interdependency  set  between  2008  and  2011.  This  is  somewhat  surprising  since,  as 
Ouyang  states,  "modeling  and  simulation  of  interdependent  CISs  become  a  critical  field  of  contemporary 
research  and  study".2 
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Figure  1.  Master  and  Interdependency  Data  Sets,  Number  of  Publications,  2003-2012 

The  data  from  the  two  sets  were  loaded  into  VantagePoint,c  a  text  mining  tool,  where  terms  were 
cleaned  and  various  subject  groupings  were  created  to  enable  analysis.  The  key  data  points  or  fields 
used  for  most  analyses  were  Subjects,  Publication  Year  and  Affiliations  (sponsoring  organizations  or 
institutions  affiliated  with  the  authors).  The  subject  field  is  an  amalgamation  of  author-supplied 


Most  annual  data  in  the  project  is  report  for  2012  as  the  data  for  2013  did  not  represent  a  complete  year  at  the  time  of  data 
collection  (October-November).  If/when  data  is  included  for  2013  it  is  based  on  percentage  of  records  in  each  year. 
c  VantagePoint  is  produced  by  U.S.  company  Search  Technology,  http://www.thevantagepoint.com/ 
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keywords,  controlled  subject  terms,  and  words  or  phrases  extracted  from  the  title  and  abstract  fields. 
These  subject  terms  were  cleaned  to  merge  singular  and  plural  forms,  to  group  words  with  very  similar 
meaning,  and  to  normalize  vocabularies  as  much  as  possible.  Two  levels  of  subject  groups  were  made: 

Subject  Groups:  subject  terms  were  organized  into  122  groups  based  on  key  topics  in  the  CIPR  field  as 
identified  by  literature  reviews,  relevant  websites,  and  conversations  with  the  client.  The  122  subject 
groups  cover  99%  of  the  entire  dataset. 

Thematic  Groups:  64  of  the  122  subject  groups  were  further  categorized  into  four  thematic  "groups  of 
groups"  to  enable  the  comparison  of  groups  with  similar  themes  and  the  detection  of  topics  with 
increasing  levels  of  research  in  the  last  five  years.  The  four  thematic  groups  are:  Modeling  and 
Simulation  Techniques,  Critical  Infrastructure,  Threats,  and  Government  Levels.  Table  12  in  Section  7.2.1 
of  this  report  depicts  the  thematic  classification  scheme  for  subject  groups  with  scope  notes  for  each 
category. 

Following  the  presentation  of  key  players  in  the  two  sets,  we  present  analyses  of  the  subject  groups  with 
a  particular  focus  on  the  Modeling  and  Simulation  group  in  which  we  examine  major  and  emerging 
trends  and  the  co-occurrence  of  specific  techniques  with  the  various  Critical  Infrastructure  and  Threats. 

For  further  study  of  CIPR  modeling  and  simulation,  the  reader  is  advised  to  consult  the  following  key 
documents  in  addition  to  those  provided  in  the  reference  section. 

Yusta  JM,  Correa  GJ,  Lacal-Arantegui  R,  Methodologies  and  applications  for  critical  infrastructure 
protection:  State-of-the-art.  Energy  Policy  2011]  39:6100-6119. 

Pederson  P,  Dudenhoeffer  D,  Hartley  S,  Permann  M.  Critical  Infrastructure  Interdependency  Modeling:  A 
Survey  of  U.S.  and  International  Research.  2006:  Idaho  National  Laboratory. 

http://www.inl.gov/technicalpublications/Documents/3489532.pdf  (Accessed  in  December  3,  2013.) 

4  FINDINGS 

4.1  Overview  of  Available  Tools 

4.1.1  Tools  in  general 

Yusta,  Correa  and  Lacal-Arantegui3  present  a  survey  of  55  articles,  reports  and  standards  between  1999- 
2010  that  propose  different  methodologies,  applications  and  software  tools  for  critical  infrastructure 
protection.  They  identified  five  different  modeling  techniques  that  use  either  a  simulation  paradigm  or 
decision-making  procedure  including:  multi-agent  systems,  system  dynamics,  rating  matrices,  relation 
databases  and  network  theory.  They  also  identified  seven  supplementary  computational  methods  and 
techniques  including  continuous  time-step  simulation,  discrete  time-step  simulation,  Monte  Carlo 
simulation,  decision  trees,  geographic  information  systems  (GIS),  risk  management  techniques  or  real 
time  record.  Many  of  these  modeling  techniques  and  supplementary  techniques  are  found  in  our 
dataset.  The  authors  further  identify  the  critical  infrastructures  that  are  being  modeled,  the  maturity 
and  availability  of  the  applications  or  platforms  and  their  risk  management  stages  (identification,  risk 
assessment,  prioritization  of  actions,  implementation  programs  and  effectiveness  measurement).  An  in 
depth  explanation  of  some  of  the  software  tools  is  provided  as  an  appendix  to  the  article. 
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Yusta,  Correa  and  Lacal-Arantegui  identified  a  number  of  trends  through  their  survey.  First,  the  authors 
noticed  a  trend  to  study  and  analyze  infrastructure  from  construction  on  to  the  current  state  in  which 
threats  are  evaluated  and  risk  management  frameworks  are  used  to  assess  vulnerabilities.  A  second 
trend  is  a  focus  on  understanding,  through  simulation,  the  dynamic  behavior  of  critical  infrastructure  to 
identify  vulnerabilities. 

Pederson,  Dudenhoeffer,  Hartley,  and  Permann4  of  Idaho  National  Laboratory  argue  that  "the  modeling 
and  analysis  of  interdependencies  between  critical  infrastructure  elements  is  a  relatively  new  and  very 
important  field  of  study"  and  conduct  a  survey  of  U.S.  and  international  critical  infrastructure 
interdependency  modeling  tools  to  provide  guidance  for  directing  R&D  to  address  gaps  in  the  field.  The 
survey  presents  30  tools,  the  Cl  sector  is  the  tool  is  used  in,  the  simulation  type  (input-output  or  agent 
based  as  well  as  continuous  or  discrete),  the  system  model  (integrated  or  coupled),  hardware/software 
requirements,  users  and  maturity  level.  In  addition  to  a  table  listing  each  tool  and  its  various  features,  a 
1-2  page  overview  of  each  tool  is  also  provided. 

4.1.2  Interdependency  and  Regional  Tools 

This  study  identified  113  specific  tools  related  to  interdependency  or  regional  modelling  (82  focused  on 
interdependency,  18  focused  on  regional  or  local,  13  focused  on  government/business  continuity 
modeling  and  simulation,  with  some  repeats  between  groups).  Eight  articles  mentioned  CIPR  exercises. 
Additional  analyses  were  performed  on  the  list  of  tools  in  section  4.4.2  (e.g.  co-occurrence  with  the 
modeling  and  simulation  techniques  as  well  as  critical  infrastructure  found  in  the  dataset).  Additionally, 
wherever  possible,  additional  tools  from  the  master  set  are  mentioned  in  analyses  throughout  the 
report. 


4.2  Key  Players 

4.2.1  Top  Affiliations  -  Master  Set 

A  breakdown  of  the  top  10  publishing  countries  in  the  master  set  is  presented  in  Figure  2. 
Unsurprisingly,  of  the  majority  of  publications  produced  by  the  top  10,  55%  originate  in  the  United 
States  (728/1313)d.  The  U.S.  is  followed  by  Italy  (164  publications)  with  Canada  (84)  in  third  place.  One 
potential  explanation  for  the  predominance  of  Italy  is  that  the  top  Italian  affiliation  (ISPRA)  is  part  of  the 
European  Commission's  Joint  Research  Centre  and  conducts  EU  level  research.  Similarly,  the  second  top 
Italian  affiliation  (ENEA)  also  works  on  Trans-European  projects,  program  and  initiatives,  including  the 
EU's  Seventh  Framework  Programme  for  Research  (FP7).  Thus  both  these  institutes,  which  together 
have  52  publications,  are  actually  representing  a  pan-European  effort. 


d  1313  is  the  total  number  of  unique  publications  produced  by  the  top  10  affiliations.  Numerous  publications  are 
affiliated  with  more  than  one  country  hence  the  addition  of  each  country's  number  of  publications  equals  1441. 
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Figure  2.  Master  Set,  Top  10  Countries 


Figure  3  lists  the  top  affiliations  (>  20  publications)  for  the  master  set  and  reflects  the  dominance  of 
American  and  Italian  players.  This  list  has  9  governmental  affiliations  and  7  academic  institutions.  A 
selection  of  the  key  affiliations  will  be  examined  in  more  detail  below. 
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Affiliation 

Sandia  National  Laboratories,  Albuquerque,  NM,  USA 
Naval  Postgraduate  School,  CA  USA 
Carnegie-Mellon  University,  Pittsburgh,  PA,  USA 
University  of  Virginia,  Charlottesville,  VA,  USA 
Idaho  National  Laboratory,  ID,  USA 
Institute  for  the  Protection  and  Security  of  the  Citizen,  Ispra,  Italy 
Swiss  Federal  Institute  of  Technology  Zurich  (ETH),  Zurich,  Switzerland 

Fraunhofer  Institutes,  Germany 

Congressional  Research  Service,  Washington,  DC.,  USA 

ENEA  (Italian  National  Agency  for  New  Technologies,  Energy  and  the 

Environment),  Rome,  Italy 

Department  of  Homeland  Security,  Washington,  DC. 
Purdue  University,  West  Lafayette,  IN,  USA 
Air  Force  Institute  of  Technology,  Wright-Patterson  AFB,  OH,  USA 
Government  Accountability  Office,  Washington,  DC. 

Third  University  of  Rome,  Rome,  Italy 
Georgia  Institute  of  Technology,  Atlanta,  GA,  USA 
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Figure  3.  Top  Affiliations  in  Master  Dataset,  >  20  Publications,  2003-2013 


Among  the  top  U.S.  governmental  affiliations  are  Sandia  National  Laboratories  (41  publications),  Idaho 
National  Labs  (30)  and  the  Department  of  Homeland  Security  (23).  Sandia  National  Laboratories  is  operated  by 
Sandia  Corporation,  a  wholly  owned  subsidiary  of  Lockheed  Martin  and  is  a  contractor  for  the  U.S.  Department 
of  Energy's  National  Nuclear  Security  Administration.  Sandia  National  Laboratories  has  an  Infrastructure 
Security  program  whose  mission  is  to  develop  and  apply  "technologies  and  analytical  approaches  to  secure  the 
nation's  critical  infrastructure  against  natural  or  malicious  disruption".  Within  this  program  is  the  National 
Infrastructure  Simulation  and  Analysis  Center  (NISAC)  which  is  co-run  with  the  U.S.  Department  of  Homeland 
Security  and  Los  Alamos  National  Laboratory.  The  NISAC  mission  is  to  "provide  strategic,  multidisciplinary 
analyses  of  interdependencies  and  consequences  of  infrastructure  disruption...".5  NISAC  has  produced  the  N- 
ABLE  tool  which  is  used  to  simulate  critical  infrastructure  interdependencies  of  businesses  in  the  U.S.  economy. 

Idaho  National  Laboratory  (INL)  is  operated  for  the  U.S.  Department  of  Energy  by  Battel le  Energy  Alliance. 

Their  Critical  Infrastructure  Protection  program  performs  research,  conducts  training  and  develops 
technologies  in  modeling  and  simulation  as  well  as  testing,  evaluation  and  demonstration  for  both  physical  and 
cyber  security  purposes.  Two  key  outputs  of  the  INL  include  the  Critical  Infrastructure  Modeling  Simulation 
(CIMS)  program  and  the  Critical  Infrastructure  Protection  and  Resilience  Simulator  (CIPRSim)  framework. 
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In  Italy,  the  top  governmental  affiliations  are  the  Institute  for  the  Protection  and  Security  of  the  Citizens 
(ISPRA,  30  publications)  and  the  Italian  National  Agency  for  New  Technologies,  Energy  and  Sustainable 
Economic  Development  (ENEA,  24).  ISPRA  is  one  of  the  seven  institutes  of  the  European  Commission's  Joint 
Research  Centre.  Research  at  ISPRA  focuses  on,  among  other  topics,  internal  security  and  disasters  and 
response.  These  research  areas  cover  prevention,  assessment,  risk  management  and  response  to  intentional, 
natural  and  technological  disasters  on  critical  infrastructures  at  the  EU  level.  ISPRA  is  also  a  thematic 
coordinator  in  the  European  Reference  Network  for  Critical  Infrastructure  Protection  (ERNCIP). 

Within  the  ENEA  is  the  Unit  for  Environment  and  Energy  Modeling  (UTMEA)  which  focuses  on,  among  other 
topics,  the  security  and  vulnerability  of  Italian  and  European  technological  networks  to  natural  and  human 
caused  disasters.  One  of  the  programs  within  this  unit  is  the  Critical  Infrastructure  Protection  and 
Interdependency  Analysis  where  a  number  of  tools  have  been  developed  including  MICIE,  IRRIIS  and  DIESIS. 
New  projects  of  potential  interest  that  are  emerging  from  this  group  include  the  Network  of  Excellence 
CIPRNet  and  CockpitCI. 

The  top  academic  institutions  are  dominated  by  the  United  States.  The  Naval  Postgraduate  School  (40 
publications)  which  is  home  to  the  Center  for  Infrastructure  Defense  focuses  on  both  military  and  civilian 
critical  infrastructure.  The  Carnegie  Mellon  University's  (39)  Center  for  Sensed  Critical  Infrastructure  Research 
(CenSCIR)  works  on  using  sensor  systems,  processes  and  technologies  to  collect  data  on  critical  infrastructure 
for  improved  decision  making.  CIPR  research  has  also  come  out  of  Carnegie  Mellon's  Software  Engineering 
Institute  in  the  form  of  OCTAVE  (Operationally  Critical  Threat,  Asset  and  Vulnerability  Evaluation)  which  is  a 
suite  of  tools  for  risk-based  information  security  strategic  assessment  and  planning.  The  University  of  Virginia's 
(34)  Center  for  Risk  Management  of  Engineering  Systems  is  an  industry  and  government  sponsored  research 
center  that  has  expertise  in  the  areas  of  risk  modeling  and  assessment,  defense  and  civil  infrastructure 
systems,  critical  infrastructure  protection,  preparedness  and  resilience  as  well  as  infrastructure 
interdependency  analysis.  Their  website  includes  an  extensive  list  of  publications  that  may  be  of  interest  for 
further  research.  One  Italian  university,  the  Third  University  of  Rome  (21,  aka  Roma  Tre  University),  appears  to 
have  CIPR  research  emerging  from  its  department  of  Informatics  and  Automation,  in  the  Faculty  of 
Engineering,  although  besides  seeing  this  reference  on  personal  Curricula  Vitae  and  doctoral  theses  in  the  field, 
this  is  hard  to  confirm  as  the  main  website  is  in  Italian  and  the  English  website  provides  extremely  limited 
information. 

As  was  seen  in  Figure  1 ,  Canada  has  a  strong  showing  in  the  dataset  with  84  publications  in  total.  The  top 
Canadian  institutions,  apart  from  Defense  Research  and  Development  Canada  (DRDC),  are  academic.  Figure  4 
lists  the  Canadian  affiliations  with  at  least  2  publications. 
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Figure  4.  Top  Canadian  Affiliations  in  Master  Dataset,  >  2  Publications,  2003-2013 

The  pre-eminent  Canadian  affiliation  is  the  University  of  British  Columbia  (UBC)  with  15  publications.  UBC  is 
one  of  six  Canadian  Universities  (along  with  York  University,  University  of  Saskatchewan,  Ecole  Polytechnique 
de  Montreal,  University  of  Toronto  and  University  of  Guelph)  that  were  involved  in  the  now  completed  Joint 
Infrastructure  Interdependencies  Research  Program  (JIIRP)  whose  goal  was  to  secure  Canada's  infrastructure 
from  threats  and  vulnerabilities  due  to  interdependencies.  One  tool  that  emerged  from  the  JIIRP  at  UBC  is  the 
Infrastructure  Interdependencies  Simulator  (l2Sim).  UBC  is  also  involved  in  the  development  of  the  Disaster 
Response  Network  Enabled  Platform  (DR-NEP)  which  is  a  tool  for  disaster  responders  and  scientific  researchers 
for  managing  responses  to  catastrophic  disasters.  This  project  is  a  collaboration  between  the  University  of  New 
Brunswick,  Western  University,  the  Asian  Institute  of  Technology  (Thailand)  and  the  Italian  ENEA. 


York  University  (9)  was  also  involved  in  the  JIIRP  project.  Some  of  York  University's  research  has  emerged  from 
the  Department  of  Earth  and  Space  Science  and  Engineering  (e.g.  use  of  Lidar  to  map  utility  corridor  objects; 
use  of  GIS  for  spatial  analysis  of  people's  vulnerabilities,  Cl  and  land  use)  and  the  Schulich  School  of  Business 
(e.g.  modeling  and  simulation  of  cyber  interdependencies  between  critical  infrastructures).  York  also  offers  a 
graduate  program  in  Disaster  and  Emergency  Management. 

The  University  of  Western  Ontario's  (UWO,  9)  Engineering  Department  focuses  on,  amongst  other  topics, 
natural  disasters  mitigation  and  management,  with  numerous  facilities  or  research  groups  including  a 
Boundary  Layer  Wind  Tunnel  Laboratory,  Geotechnical  Research  Centre,  Water  Resources  and  Systems 
Modeling  Group  and  the  Institute  for  Catastrophic  Loss  Reduction.  Other  research  focuses  on  infrastructure 
renewal  which  includes  systems  modeling  and  natural  disaster  mitigation  as  well.  UWO's  role  in  the  DR-NEP  is 
related  to  the  identification  of  ontological  models  of  its  critical  Infrastructure  and  disaster  response  entities 
into  a  common  representation  as  well  as  the  simulation  of  real-time  disasters  on  its  University  campus. 
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Although  the  Ecole  Polytechnique  de  Montreal  is  known  to  be  active  in  the  field,  it  only  has  3  publications  in 
the  dataset.  An  additional  search  was  conducted  in  the  main  bibliographic  databases  used  in  this  study  to 
confirm  that  publications  were  not  missed.  A  further  examination  of  the  website  also  confirmed  that  some  of 
the  additional  publications  that  are  emerging  from  Ecole  Polythechnique's  Centre  Risque  &  Performance,  are 
not  available  through  bibliographic  databases  and  therefore  did  not  make  it  into  this  report. 


4.2.2  Top  Affiliations  -  Interdependency 

The  top  affiliations  for  the  Interdependency  set  closely  resemble  the  master  set  albeit  with  much  lower 
numbers  of  publications.  Figure  5  shows  the  top  10  affiliations  (>  7  publications).  Because  the  threshold  is 
lower  (>  7  as  opposed  to  >  20),  a  number  of  new  affiliations  appear  on  this  list  including:  Polytechnic  University 
of  Milan  (Italy),  Huazhong  University  of  Science  and  Technology  (China),  University  Campus  Bio-Medico  of 
Rome,  Rice  University  (U.S.),  the  Italian  National  Research  Council  (CNR),  TNO  Defense  (Netherlands)  and 
Gjpvik  University  College  in  Norway. 
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Figure  5.  Top  Affiliations  in  Interdependency  Dataset,  >  7  Publications,  2003-2013 
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Of  particular  interest  is  the  University  Campus  Bio-Medico  of  Rome  (11),  which  is  home  to  the  Complex 
Systems  and  Security  (COSERITY)  Lab,  one  of  the  leading  Italian  research  institutes  on  Homeland 
Security  and  Critical  Infrastructure  Protection.  COSERITY  is  currently  focused  on  interdependency 
models  and  fault  detection  and  reaction  strategies  to  cyber  and  stealth  attacks.  The  Dr.  Duenas-Osorio 
Research  Group  at  Rice  University  (8)  focuses  on  computational  and  theoretical  models  for  structure 
and  infrastructure  system  reliability  and  risk  assessment  (SISRRA)  in  the  context  of  natural  hazards, 
deterioration,  and  complex  operation.  TNO  Defense  (7)  is  involved  in  the  Critical  Infrastructure 
Preparedness  and  Resilience  Research  Network  (CIPRNet)  and  has  worked  on  the  DIESIS  tool  and  the 
Integrated  Risk  Reduction  of  Information-based  Infrastructure  (IRRIS)  tool.  With  DIESIS,  TNO  Defense  is 
responsible  for  developing  a  modeling  and  simulation  tool  for  dependent  Cl.  With  IRRIS,  it  participated 
in  the  development  of  the  Middleware  Improved  Technology  (MIT)  software  and  Simulation  for  Critical 
Infrastructure  Protection  (SimCIP)  tool.  TNO  Defense  has  co-published  a  Good  Practices  Manual  for  CIP 
Policies  as  well  as  many  other  CIP  related  publications. 

Figure  6  shows  the  top  Canadian  affiliations  in  the  interdependency  set.  Once  again,  UBC,  York  and 
Western  are  at  the  top  of  the  list. 


Figure  6.  Top  Canadian  Affiliations  in  Interdependency  Dataset,  >  2  Publications,  2003-2013 
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4.3  Major  Topics 

As  described  above,  122  subject  groups  were  created  in  the  master  set  to  group  together  topics  based 
on  terms  in  the  various  keyword  fields  and  from  words  in  titles  and  abstracts.  These  groupings  were 
created  based  on  a  review  of  the  literature  in  the  field,  relevant  websites,  and  conversations  with  the 
client.  64  of  the  122  subject  groups  were  further  categorized  into  four  thematic  "groups  of  groups"  to 
enable  comparison  of  groups  with  similar  content  and  the  detection  of  topics  with  increasing  levels  of 
research  in  the  last  five  years.  The  four  thematic  groups  are:  Modeling  and  Simulation  Techniques, 
Critical  Infrastructure,  Threats,  and  Government  Levels.  These  thematic  groups  are  used  to  drill  down 
into  the  data  to  answer  the  key  questions.  The  development  of  these  thematic  groups  will  be  discussed 
in  more  detail  below. 

Cluster  analysis  based  on  the  co-occurrence  of  keywords  in  bibliographic  records  is  one  method  of 
visualizing  and  analyzing  topics  that  are  important  in  a  scientific  and  technical  domain  as  well  as  their 
relationships  to  each  other.  Figure  7  shows  details  of  the  center  of  a  cluster  map  based  on  the  122 
subject  groups  created  in  the  project,  generated  using  TouchGraph  Navigator6 software.  TouchGraph's 
clustering  algorithm  clusters  terms  together  based  on  statistical  similarity  to  each  other  (i.e.  word  co¬ 
occurrences)  and  dissimilarity  with  other  clusters.  Generally,  a  cluster  illustrates  a  self-contained  group 
of  concepts  that  is  independent  from  (though  still  connected  to)  the  rest  of  the  graph.  The  size  of  the 
nodes  in  this  map  represents  the  relative  number  of  publications  associated  with  each  node  and  the 
lines  in  between  nodes  show  the  correlation  coefficient  (multiplied  by  100)  between  two  nodes.  Only 
correlations  of  21%  or  greater  are  shown  below/ 


TouchGraph  Navigator  is  produced  by  the  US  company  TouchGraph  LLC:  http://www.touchgraph.com/navigator 
f  Correlation  values  are  set  at  a  percentage  that  makes  the  graph  readable  while  still  showing  a  good  amount  of  connection 
between  clusters.  It  can  vary  for  every  map  as  it  is  determined  through  expert  judgment  as  well  as  trial  and  error;  however  20% 
is  the  authors'  standard  starting  point. 


NRC 


Page  17  of  63 


Knowledge  Management 


CIPR  Literature  Survey:  Modeling  and  Simulation 


Page  18  of  63 

NRC  Knowledge  Management 


CIPR  Literature  Survey:  Modeling  and  Simulation 


December  2013 


The  majority  of  terms  on  the  map  fall  into  one  of  the  four  thematic  groups  and  either  represent  a 
modeling  and  simulation  technique,  a  critical  infrastructure,  a  threat  or  a  government  body.  In  addition, 
we  see  other  higher  level  concepts  related  to  the  field  such  as  can  be  seen  in  the  bright  green  cluster 
which  includes  resilience,  economic  issues,  mitigation  and  recovery.  Another  similar,  but  smaller,  cluster 
in  purple  just  below  includes  risk  assessment,  measurement  and  engineering. 

In  the  center  of  this  map  is  an  orange  cluster  representing  the  largest  critical  infrastructure  groups  in  the 
dataset  and  includes  Energy  and  Utilities,  ICT,  Transportation  and  Water.  In  the  same  cluster  is  Coastal 
zones  which  correlates  with  Transportation  due  to  ports  and  the  transportation  of  goods  through  ports; 
environmental/ecological  topics  correlates  with  Water. 

Above  and  to  the  right  is  a  red  cluster  that  includes  Safety  and  Health  Care  (critical  infrastructure 
groups)  along  with  Disease/pan/epidemic  (a  threat  to  public  health  and  safety)  and  then  two  groups, 
National  and  Department  of  Homeland  Security  that  are  categorized  into  the  government  thematic 
category.  This  is  likely  occurring  because  the  group  Safety  includes  such  keywords  as  public  safety, 
emergency  preparedness  and  management,  dams  and  levees,  as  well  as  CBRNE  related  terms. 

To  the  right  is  a  yellow  cluster  that  includes  Cyber,  SCADA,  Terrorism,  Government,  Manufacturing  and 
CBRNE  and  Sept  11.  Manufacturing  includes  the  chemical  industry  and  the  defense  industrial  base, 
hence  the  connection  to  CBRNE. 

Many  of  the  nodes  further  out  from  the  center  (not  shown  in  graph)  represent  the  various  modeling  and 
simulation  techniques  that  are  found  in  the  dataset.  While  few  of  them  cluster  with  the  critical 
infrastructure  or  threats  in  this  map,  multiple  co-occurrence  maps  of  these  techniques  will  be  presented 
in  section  4.3.1  for  the  master  set  and  section  4.4.1  for  the  interdependency  set  further  down. 

Following  the  development  of  the  cluster  map,  the  four  thematic  groups  were  created  in  order  to  gain 
more  insight  into  how  the  various  modeling  and  simulation  techniques  were  being  used. 

A  thematic  group  for  the  critical  infrastructure  was  created  based  on  the  ten  Canadian  critical 
infrastructure  sectors  that  are  listed  in  the  Canadian  National  Strategy  for  Critical  Infrastructure.1 
Guidance  for  adding  various  groups  to  each  of  these  10  sectors  was  gained  from  the  PTSC-Onlinee 
website6  and  is  provided  in  Table  2  .  Additional  similar  and/or  related  terms  found  in  the  dataset  were 
also  added  to  each  Cl  sector  as  appropriate. 


6  PTSC-Online  is  a  Canada  wide  virtual  on  line  community  which  networks  emergency  management  personnel  and  emergency 
services  for  the  purpose  of  continuously  improving  the  safety  and  resiliency  of  our  Canadian  communities. 
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Table  2.  Canada's  10  Critical  Infrastructure  Sectors. 

Cl  Sectors 

Examples  of  Components 

Energy  and  Utilities 

Electrical  power,  natural  gas,  oil  production  and  transmission  systems 

Information  &  Communication 

Telecommunications,  broadcasting  systems,  software,  hardware  and 

Technology  (ICT) 

networks  including  the  Internet 

Finance 

Banking,  securities  and  investment 

Health  Care 

Hospitals,  Health  Care  and  blood  supply  facilities,  laboratories  and 
pharmaceuticals 

Food 

Safety,  distribution,  agriculture  and  food  industry 

Water 

Drinking  water  and  wastewater  management 

Transportation 

Air,  rail,  marine,  surface  and  mass  transit 

Safety 

Chemical,  biological,  radiological  and  nuclear  safety,  hazardous 
materials,  search  and  rescue,  emergency  services  and  dams 

Government 

Service  delivery,  facilities,  information  networks,  assets  and  key 
national  sites  and  monuments 

Manufacturing 

Defense  industrial  base,  chemical  industry 

In  addition  to  these  10  sectors,  the  following  terms  were  added  to  the  thematic  group  due  either  to 
their  presence  in  the  U.S.  list  of  critical  infrastructure  sectors  or  their  frequency  in  the  dataset:  Cyber, 
Buildings,  Military,  Public  infrastructure,  Coastal  zones.  Cyber  was  given  its  own  group  because  there 
were  so  many  publications  that  focused  specifically  on  cyber  issues  and  less  than  half  co-occurred  with 
ICT  thus  arguably  warranting  its  own  examination.  Buildings  was  given  its  own  group  because  they  are  a 
physical  infrastructure  that  can  fall  under  threat  but  did  not  specifically  fit  into  any  other  group.  While 
the  Cl  group  Government  does  allow  for  the  addition  of  facilities,  key  national  sites  and  monuments, 
many  of  which  are  buildings,  no  specific  governmental  facilities,  sites  or  monuments  were  explicitly 
found  in  the  dataset  thus  further  calling  for  a  Cl  group  to  capture  the  generic  buildings  found  in  the  set. 

Figure  8  displays  the  number  of  records  between  the  years  of  2003-2012  for  each  critical  infrastructure 
with  the  total  for  each  also  provided  (totals  are  for  2003-2014).  It  is  important  to  note  that  these 
numbers  are  not  mutually  exclusive  in  that  a  publication  that  is  categorized  in  the  Energy  and  Utilities 
sector  may  also  be  categorized  in  the  ICT  sector  because  the  publication  discusses  both  sectors.  For 
instance,  223  of  the  total  711  Energy  and  Utilities  are  also  found  in  ICT. 

Since  the  overall  dataset  dropped  somewhat  in  2012  (as  displayed  in  Figure  1 )  it  is  not  surprising  that 
the  2012  number  of  records  are  slightly  lower  for  quite  a  few  of  the  sectors.  Exceptions  include  Energy 
and  Utilities,  Safety,  Water,  Health  Care,  Military  and  Public  infrastructure  which  continue  to  rise  in 
2012.  Manufacturing  and,  arguably,  the  Food  and  Government  sectors  show  a  more  notable  drop  in 
2012. 
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Critical  Infrastructure 

Total 

2003 

2004 

2005 

2006 

Year 

2007 

2008 

2009 

2010 

2011 

2012 

Energy  and  Utilities 

711 

• 

17 

#  35 

#47 

•  33 

•  73 

# 54 

0  88 

•97 

•13 

^^14 

ICT 

611 

• 

16 

0  41 

•  43 

43 

62 

#61 

74 

85 

88 

074 

Safety 

488 

• 

13 

21 

34 

#  36 

#  53 

•  37 

#  46 

#  55 

074 

84 

Transportation 

479 

• 

11 

22 

35 

30 

•  51 

25 

62 

#67 

81 

•  73 

Cyber 

348 

• 

11 

14 

•  28 

23 

#  33 

•  33 

#  38 

#  41 

(  |  60 

#  53 

Government 

287 

• 

15 

19 

24 

26 

33 

34 

22 

•  32 

#  45 

28 

Water 

285 

• 

12 

13 

18 

12 

$  35 

14 

33 

43 

#  36 

0  58 

Health  Care 

232 

6 

9 

14 

18 

28 

26 

20 

•  31 

31 

•  34 

Buildings 

172 

• 

3 

4 

11 

10 

18 

6 

15 

24 

#  36 

28 

Military 

172 

• 

9 

13 

12 

7 

•  27 

16 

10 

28 

26 

•  27 

Finance 

151 

3 

9 

16 

13 

21 

27 

12 

14 

16 

14 

Manufacturing 

141 

3 

12 

12 

16 

23 

10 

15 

19 

15 

10 

Public  Infrastructure 

129 

7 

2 

12 

10 

11 

11 

11 

14 

15 

23 

Coastal  Zone 

82 

2 

10 

6 

7 

3 

12 

15 

14 

11 

Food 

59 

2 

4 

3 

3 

6 

8 

5 

10 

11 

6 

Figure  8.  Critical  Infrastructure  Types,  Numbers  of  Publications  by  Year,  2003-2012 

The  U.S.  Presidential  Policy  Directive  21  “identifies  the  Energy  Sector  as  uniquely  critical  because  it 
provides  an  "enabling  function"  across  all  critical  infrastructure  sectors".7  This  likely  explains  the  high 
rates  of  research  and  development  that  we  see  here.  Additionally,  the  U.S.  Department  of  Energy  is  the 
sector  specific  agency  responsible  for  information  sharing  and  coordination  of  exercises  that  address 
energy  infrastructure  issues8  and  as  was  seen  in  the  affiliations  section,  is  ultimately  behind  much  of  the 
CIP  research  that  is  emerging  from  National  Labs  such  as  Sandia,  Idaho  and  others.  Similarly 
organizations  such  as  the  North  American  Electric  Reliability  Corporation  (NERC),  which  is  a  not-for- 
profit  entity  whose  mission  is  to  ensure  the  reliability  of  the  power  system  in  North  America  (including 
the  United  States,  Canada  and  the  northern  portion  of  Baja  California,  Mexico)  is  very  active  in  Cl 
protection  and  resilience  and  has  issued  numerous  reliability  and  critical  infrastructure  protection 
standards. 

The  Water  sector  has  seen  fairly  steady  growth  since  about  2009  which  is  shortly  after  the  release  of  the 
2007  Water:  Critical  Infrastructure  and  Key  Resources  Sector-Specific  Plan  as  input  to  the  National 
Infrastructure  Protection  Plan9  which  set  out  new  R&D  priorities  for  water  infrastructure  protection. 
Some  of  these  priorities  include:  updating  identification,  prioritization  and  understanding  of  physical  and 
cyber  threats,  and  their  consequences,  along  with  vulnerabilities  to  drinking  water  and  wastewater 
infrastructure;  improving  analytical  methodologies  and  monitoring  systems  for  drinking  water;  and 
addressing  infrastructure  interdependencies  and  planning  for  contingencies. 

A  second  thematic  group  representing  the  threats  to  critical  infrastructure  was  also  created.  This 
thematic  group  was  formed  by  grouping  all  the  threat-related  terms  that  were  identified  in  the  122 
subject  group  creation.  A  total  of  15  threats  were  identified  and  the  number  of  records  in  each  year  can 
be  seen  in  Figure  9  (totals  include  2013-2014).  Again,  there  is  an  overall  trend  to  see  a  slight  drop  in 
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2012.  Notable  anomalies  include  the  significant  peak  in  CB/?/V£- related  articles  in  2011  (22  as  compared 
to  8  in  2010  and  12  in  2012),  as  well  as  for  Climate  change  (19  in  2011  as  compared  to  6  in  2010). 

Year 


Threat 

Total 

2003 

2004 

2005 

2006 

2007 

2008 

2009 

2010 

2011 

2012 

Terrorism 

308 

17 

• 

21 

^38 

+ 

37 

^40 

(  30 

0 

29 

•27 

^36 

29 

Disasters 

263 

• 

4 

• 

8 

• 

16 

• 

16 

•27 

18 

0: 

38 

36 

46 

32 

Earthquake 

116 

• 

4 

• 

1 

• 

5 

• 

6 

4 

6 

•  17 

•  19 

•  28 

•  20 

CBRNE 

109 

• 

1 

• 

4 

• 

8 

14 

•  9 

12 

•  11 

•  8 

•  22 

12 

Natural  disaster 

108 

• 

3 

• 

3 

• 

6 

• 

7 

13 

6 

• 

15 

•  18 

•  16 

11 

Floods 

96 

• 

1 

• 

2 

• 

1 

• 

5 

5 

•  6 

• 

5 

14 

•  21 

•  27 

Fire 

70 

• 

2 

• 

2 

• 

6 

• 

4 

9 

00 

• 

0 

3 

0  7 

10 

0  11 

Extreme  weather 

51 

• 

1 

• 

1 

• 

4 

• 

3 

2 

2 

• 

4 

7 

12 

11 

Flurricane 

61 

• 

1 

• 

4 

•  7 

00 

• 

• 

9 

10 

10 

10 

Blackout 

36 

• 

1 

• 

3 

• 

2 

6 

4 

• 

7 

3 

6 

4 

Disease/  pan/epidemic 

40 

• 

3 

• 

7 

4 

5 

• 

5 

6 

4 

5 

Climate  change 

52 

1 

4 

• 

5 

CD 

• 

•  19 

•  12 

Denial  of  service  Attack 

33 

• 

4 

• 

1 

• 

5 

3 

• 

2 

4 

5 

6 

Tsunami 

20 

2 

1 

1 

7 

5 

2 

Drought 

4 

1 

3 

Figure  9.  Master  set,  Threat  by  Year,  2003-2012 

Given  that  the  threat  labeled  as  Disasters  is  quite  vague  and  all-encompassing  with  regards  to  CIPR,  a 
few  additional  steps  were  taken  to  improve  clarity.  First,  an  additional  group  called  Natural  disasters 
was  extracted  from  the  larger  Disasters  group.  This  group  was  created  by  gathering  together  all  terms 
that  explicitly  include  the  words  "natural  disaster".  Next,  a  co-occurrence  map  was  run  on  the  Disasters 
group  with  all  the  other  threats  in  the  dataset  to  see  which  specific  types  of  disasters  are  specified  in  the 
Disasters  group.  Figure  10  shows  the  results  and  indicates  that  Natural  disasters  are  the  most  frequently 
specified  type  of  disaster  and  is  mentioned  70  times  in  the  263  records  that  are  categorized  as  Disasters. 
These  results  show  a  very  similar  ordering  (based  on  total  number  of  records)  to  the  main  ordering  of 
threats  in  the  master  set. 
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Threat 

Natural  Disaster 

Terrorism 

Earthquake 

Floods 

Flurricane 

CBRNE 

Fire 

Extreme  weather 

Disease/  pan/epidemic 

Tsunami 

Blackout 

Climate  change 

Denial  of  service  Attack 

Drought 


Disasters 


I 


43 

34 

31 

30 

21 

18 

12 

10 

8 

7 

7 

2 

1 


Figure  10.  Master  set,  Number  of  Co-occurrence  between  Disasters  and  Remaining  Threats 

4.3.1  Modeling  and  Simulation  Techniques 

The  most  relevant  thematic  group  in  this  project  is  the  Modeling  and  Simulation  (M&S)  group,  as  the 
deeper  analyses  are  all  based  on  it.  This  group  comprises  27  different  techniques  or  methodologies 
found  in  the  dataset  that  relate  to  the  act  of  modeling  or  simulation.  These  groups  were  generally 
formed  by  grouping  the  main  term  (e.g.  dynamic)  with  terms  such  as  analysis,  model,  simulation, 
approach,  indices,  exercise,  methods,  examples,  schemes,  as  appropriate  for  each  M&S  technique.  It  is 
important  to  note  that  these  groups  are  not  mutually  exclusive,  in  that  publications  that  are  categorized 
in  the  Dynamic  M&S  group  may  also  be  categorized  in  one  of  the  other  techniques.  For  instance, 
Dynamic  M&S  co-occurs  with  System  Dynamics  Modeling,  Estimation,  Input-Output  Modeling,  Agent- 
based  modeling,  Classification/Pattern  Identification  and  Statistical/Numerical  techniques  at  least  ten 
times  each.  The  27  M&S  techniques,  along  with  the  total  number  of  publications  for  each,  are  listed  in 
Table  3. 
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Table  3.  Modeling  and  Simulation  Techniques,  Total  Number  of  Publications 


No. 

No. 

M&S  Technique 

Publications 

M&S  Technique 

Publications 

Dynamic  M&S 

158 

Fuzzy 

37 

Estimation 

145 

Behavioral  Analysis 

33 

Statistical/Numerical 

techniques 

124 

Bayesian 

31 

Classification/Pattern 

Identification 

112 

Systems  Dynamic  Modeling 

29 

GIS 

79 

Monte  Carlo 

29 

Graph  Theory/Model 

79 

Tree  Analysis 

26 

Topological 

68 

Clustering 

25 

Agent-based 

60 

Petri  Nets 

19 

Input-Output  Modeling 

47 

Markov 

18 

Hierarchical  Methods  (other) 

45 

Network  Theory 

13 

Game  Theory 

44 

Hierarchical  Flolographic 
Modeling 

8 

Uncertainty  Analysis 

43 

Dynamic  Input-Output 
Modeling 

8 

Forecasting 

42 

Quantitative  Vulnerability 
Modeling 

6 

Stochastic  Modeling 

41 

4.3. 1.1  Emerging  Research  Trends  in  Modeling  and  Simulation 

In  order  to  identify  emerging  research  trends  in  the  modeling  and  simulation  of  CIPR,  an  R&D 
momentum  analysis  was  conducted.  Further  explanation  of  the  methodology  behind  the  momentum 
indicator  is  included  in  Section  7.3,  but  essentially  it  plots  the  standard  deviation  of  standardized 
measures  of  publication  counts  and  velocity  (the  rate  of  publication  increase)  on  two  axes.  Nodes  which 
plot  to  the  left  of  the  Y-axis  intersection  have  below-average  velocity,  and  those  found  below  the  X-axis 
have  relatively  smaller  publication  counts.  A  third  dimension  is  added  by  sizing  nodes  relative  to  the 
total  number  of  underlying  publications.  Even  a  small  node  which  plots  to  the  right/lower  side  of  the 
axes  may  be  of  interest,  since  emerging  topics  are  typically  small  in  numbers  as  they  begin  to  attract 
research  attention  and  increase  in  velocity.  Figure  11  plots  the  momentum  of  all  27  modeling  and 
simulation  techniques. 
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Growth  rate  (velocity) 


•  Dynamic  M&S 

•  Statistical/numerical  techniques 

•  GIS 

•  Topological 

•  Input-Output  Modeling 

•  Game  theory 

•  Uncertainty  Analysis 

•  Fuzzy 

•  Bayesian 

•  System  dynamics  modeling 

•  Clustering 
Markov 

Dynamic  Inoperability  Input-Output  Model  (DIIM) 
Quant  Vulner  Ass'mnt 


•  Estimation 

•  Classification/Pattern  ID 

•  Graph  theory/model 

•  Agent  based 

•  Hierarchical  Methods  (other) 

•  Forecasting 

•  Stochastic  modeling 

•  Behavioral  Analysis 

•  Monte  Carlo 

•  Tree  Analysis 

•  Petri  Nets 

•  Network  theory 

Hierarchical  Holographic  Modeling 


Figure  11.  R&D  Momentum,  Modeling  and  Simulation  Techniques,  Master  Set,  2008-2012h 

The  majority  of  M&S  techniques  fall  on  or  below  the  x-axis.  M&S  that  are  already  "hot"  in  the  field  with 
high  number  of  records  and  a  high  rate  of  acceleration  include  Dynamic  M&S,  Estimation, 
Statistical/numerical  techniques,  GIS,  Graph  theory/model  and  Uncertainty  analysis.  Established  trends 
in  the  field  can  be  seen  in  the  use  of  Classification/pattern  analysis.  Topological  and  Agent-based 
techniques.  In  order  to  assess  whether  the  remaining  M&S  are  brand  new  (in  terms  of  application  in  the 
field  of  CIPR),  disappearing  or  emerging  topics  in  relation  to  CIPR,  the  number  of  publications  in  each 
node  in  the  lower  quadrants  is  presented  for  the  past  5  years.1 


Note  that  Game  Theory  and  Stochastic  Modeling  show  up  in  the  exact  same  place  (large  green  and  orange  circle  at  X,  -0.5,  Y, 

0). 

1  The  R&D  momentum  and  the  figures  11-13  are  based  on  data  between  2008  and  2012  because  2013  was  not  a  complete  year 
at  the  time  of  data  collection  and  would  throw  off  the  growth  rates  if  included. 
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Figure  12  presents  the  M&S  nodes  found  in  the  lower  left  quadrant,  which  is  where  the  "new"  or 
"disappearing"  techniques  in  the  field  are  typically  found.  Keeping  in  mind  that  the  overall  master  set 
saw  a  drop  in  number  of  records  in  2012,  there  are  a  number  of  things  to  point  out.  The  steadiest 
growth  can  be  seen  in  the  use  of  Bayesian  techniques.  Bayesian  techniques  have  been  used  in  the 
development  of  a  framework  for  assessing  risk  probability  with  a  particular  focus  on  information 
systems,10  in  the  use  of  max-propagation  in  a  Bayesian  networks  to  identify  critical  components  in 
infrastructure  systems,11  in  the  development  of  a  terrorist  activity  prediction  model  (TAPM)  using 
Dynamic  Bayesian  Network  (DBN)12  and  as  a  novel  approach  to  Cl  interdependency  analysis  based  on 
DBN  formalism13  to  name  just  a  few.  Fuzzy  techniques  may  also  be  rising  as  there  is  a  peak  in  2011  but 
more  data  would  need  to  be  seen  over  time  to  confirm  this.  DIIM  and  Quantitative  Vulnerability 
Assessment  (QVA)  appear  to  be  more  on  the  end  of  disappearing  rather  than  brand  new. 

Year 


Modeling  and  Simulation  Technique 

2008 

2009 

2010 

2011 

2012 

Bayesian 

• 

1 

• 

5 

• 

4 

• 

6 

• 

8 

Dynamic  Inoperability  Input-Output  Model  (DNM) 

• 

1 

• 

2 

Forecasting 

• 

12 

• 

4 

• 

8 

• 

5 

Fuzzy 

• 

3 

• 

3 

• 

6 

• 

10 

• 

5 

Hierarchical  Holographic  Modeling 

• 

1 

• 

1 

Hierarchical  Methods  (other) 

• 

4 

• 

7 

• 

9 

• 

6 

• 

5 

Input-Output  Modeling 

• 

5 

• 

4 

• 

8 

• 

2 

• 

6 

Markov 

• 

2 

• 

5 

• 

1 

• 

2 

• 

4 

Quantitative  Vulnerability  Assessment 

• 

2 

• 

2 

System  Dynamics  Modeling 

• 

1 

• 

3 

• 

4 

• 

6 

• 

4 

Figure  12.  Master  Set,  Modeling  and  Simulation  Techniques  by  Year,  Nodes  in  the  Lower  Left  Quadrant  -  New  or 

Disappearing  Techniques. 

Figure  13  represents  the  M&S  techniques  that  fall  in  the  lower  right  quadrant,  or  the  "emerging" 
techniques  quadrant.  Most  notably  we  can  see  a  doubling  in  the  use  of  Behavioural  Analysis  and  Monte 
Carlo  techniques  for  modeling  or  simulating  CIPR  in  the  last  2  years  and  a  significant  jump  in  Tree 
Analysis  in  2012. 

Year 


Modeling  and  Simulation  Technique 

2008 

2009 

2010 

2011 

2012 

Behavioral  Analysis 

• 

1 

• 

6 

• 

4 

• 

4 

•  • 

Monte  Carlo 

• 

1 

• 

3 

• 

2 

• 

5 

(J)  10 

Tree  Analysis 

• 

1 

• 

4 

• 

3 

• 

1 

•  12 

Clustering 

• 

1 

• 

4 

• 

7 

•  5 

Petri  Nets 

• 

1 

• 

3 

• 

2 

•  4 

Figure  13.  Master  Set,  Modeling  and  Simulation  Techniques  by  Year,  Nodes  in  the  Lower  Right  Quadrant  - 

Emerging  Techniques. 
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Behavioral  analysis  can  be  used  to  study  reduction  in  risk  as  a  result  of  changing  behaviors  prior  to  an 
event,  for  studying  anomalous  behaviors,  or  in  predicting  terrorists'  behavior.  Behavioral  analysis  has 
been  used  to  increase  the  resilience  of  SCADA  systems,14  cyber  threats,15  and  a  novel  application  of 
Evidential  Reasoning  to  Threat  Assessment.16  Monte  Carlo  techniques  have  been  used  as  part  of  a  Fuzzy 
Monte  Carlo  Simulation  approach  to  study  smart  grids17  in  seismic  modeling18'20  and  in  the  Sandia 
developed  Scenario  Toolkit  and  Generation  Environment  (STAGE)  simulation  engine.21  A  number  of  the 
2012  articles  on  Monte  Carlo  techniques  also  mention  a  tree  analysis  technique. 18,22,23  Most  notably, 
Tree  analysis  jumped  significantly  in  2012  with  12  records  as  compared  to  a  high  of  four  in  2009.  This 
may  indicate  that  there  was  a  recent  breakthrough  in  the  use  of  tree  analysis.  Vintr,  Valis  and  Malach 
explain  that  attack  tree-based  evaluation  of  vulnerability  is  an  analytic  method  that  has  been 
dynamically  developed  and  has  recently  had  its  theoretical  boundary  conditions  for  practical 
applications  worked  out.24  Attack  tree,  fault  tree  or  regression  tree  analysis  have  been  used  in 
Probabilistic  Seismic  Hazard  Analysis,25,26  drinking  water  treatment  plants,27  dam  security  models,28  the 
Florida  Public  Hurricane  Loss  Model,29  and  safety  of  railway  tunnels22  to  name  a  few. 

4.3. 1.2  M&S  and  Threats 

To  further  explore  the  modeling  and  simulation  techniques  that  are  discussed  in  the  dataset  a  co¬ 
occurrence  table  was  created  to  cross  reference  the  M&S  techniques  with  the  threats  that  were 
presented  in  Figure  9.  Figure  14  identifies  which  technique  is  most  frequently  used  with  each  threat. 
Some  notable  points  that  can  be  seen  in  this  figure  include  the  fact  that  Dynamic  M&S  is  most 
frequently  used  in  modeling  Disasters  (28),  Earthquakes  (22),  Natural  disasters  (19)  and  Terrorism  (17). 
Dynamic  M&S,  represents  a  group  of  general  modeling  or  simulation  techniques  that  are  referred  to  as 
dynamic  when  they  incorporate  time-dependent  parameters.30  Estimation  is  most  frequently  used  for 
Floods  (23)  with  Earthquakes  (16),  Terrorism  (15)  and  Climate  change  (13).  In  fact,  Estimation  is  the  most 
frequently  used  technique  for  all  of  Flood  AND  Climate  change  modeling.  Estimation  may  be  used  to 
model  risks,  vulnerabilities,  reliability,  costs,  damages  or  loss  etc...  GIS  is  most  frequently  used  for 
Disasters  (27),  Floods  (15),  Fire  and  Terrorism  (10  each).  GIS  is  the  most  frequently  used  method  for  all 
Fire  modeling.  GIS  is  used  in  a  variety  of  ways  and  sometimes  in  combination  with  other  M&S 
techniques  to  include  a  geospatial  understanding  of  critical  infrastructure  vulnerabilities  and  risk 
prevention.  For  example,  a  GIS-based  software  application  called  E3R  was  created  to  merge  the  needs  of 
risk  assessment,  risk  reduction  and  risk  management  during  different  phases  of  disaster  response  into  a 
single  tool.31  A  GIS  based  macro-simulation  model  was  developed  to  support  evacuation  planning  and 
training  in  the  event  of  a  terrorist  attack  on  sports  stadium.32  GIS  was  also  recently  used  in  a  Canadian 
study  to  create  vulnerability  indicators  based  on  spatial  and  attribute  data  related  to  human 
vulnerability,  critical  infrastructure  and  land  use.33  Statistical/numerical  technigues  are  used  most  for 
modeling  Disasters  (17),  Terrorism  (13)  and  CBRNE  (13).  Once  again,  Statistical / numerical  techniques  is 
the  most  frequently  used  technique  for  all  of  CBRNE  and  Disease/pan/epidemic  modeling.  One 
interesting  outlier  to  draw  attention  to  is  the  disproportionate  uses  of  Game  Theory  (15)  in  Terrorism 
modeling.  These  15  records  are  relatively  evenly  dispersed  across  the  timeframe  of  the  study.  These 
records  discuss,  amongst  other  topics,  the  use  of  Game  Theory  to  model  cyber-attacks,34  to  model  or 
simulate  terrorist  attacks  on  transportation  networks,12,35,36  to  monitor  pipelines  against  terrorist 
attacks,37  and  as  an  Adaptive  Two-Player  Hierarchical  Holographic  Modeling  (HHM)  Game  to  track 
terrorism  scenarios.38 
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4.3. 1.3  M&S  Techniques  for  Specific  Critical  Infrastructure  Types 

Figure  15  presents  a  co-occurrence  matrix  of  M&S  techniques  with  the  critical  infrastructure  types  presented  in  Figure  8. 
The  most  frequently  used  technique  for  each  Cl  is  listed  in  Table  4. 

Table  4.  Most  Frequently  Used  M&S  for  each  Critical  Infrastructure 


Critical  Infrastructure 

M&S  Technique  (No.  pubs) 

Energy  and  Utilities 

Dynamic  M&S  (51),  Estimation  (50) 

ICT 

Dynamic  M&S  (46),  Classification/Pattern  ID  (38) 

Transportation 

Dynamic  M&S  (41),  Classification/Pattern  ID  (27) 

Safety 

Estimation  (45) 

Water 

Estimation  (29) 

Cyber 

Classification/Pattern  ID  (20) 

Health  Care 

Statistical/numerical  techniques  (18) 

Buildings 

Dynamic  M&S  (24),  Classification/Pattern  ID  (13) 

Finance 

Dynamic  M&S  (13)  Input-Output  Modeling  (10) 

Government 

Dynamic  M&S  (9),  Forecasting  (9), 
Classification/Pattern  ID  (9) 

Manufacturing 

Dynamic  M&S  (12),  Estimation  (9) 

Military 

Statistical/numerical  techniques  (9) 

Public  Infrastructure 

Dynamic  M&S  (9),  Estimation  (9),  GIS  (9) 

Coastal  zone 

GIS  (11) 

Food 

Dynamic  M&S  (4),  Estimation  (4), 

Classification/Pattern  ID  (4) 

As  can  be  seen  in  Table  4,  Dynamic  M&S  is  the  top  modeling  and  simulation  technique  for  many  of  the  critical 
infrastructure  in  this  study.  Outside  of  the  top  5  techniques  used  by  all  the  critical  infrastructure,  including  Dynamic 
M&S  (158),  Estimation  (145),  Statistical/numerical  techniques  (124),  Classification/pattern  identification  (112)  and  GIS 
(79)  along  with  the  prevalence  of  each  technique  in  Energy  and  Utilities  (which  has  the  greatest  number  of  records  in 
total)  there  are  some  other  interesting  observations  that  can  be  made  from  Figure  15.  In  particular  there  is  a  significant 
use  of  Topological  methods  (40)  in  Energy  and  Utilities  and  ICT  (33)  and  of  Hierarchical  methods  (22)  used  in  ICT. 
Topological  methods  can  be  considered  as  a  type  of  network  based  methods  that  do  not  consider  node  heterogeneity, 
whereas  Hierarchical  methods  model  risks  or  vulnerabilities  from  a  levels  perspective.  System  Dynamic  Modeling  (12), 
which,  like  Dynamic  M&S,  models  complex  systems  over  time,  is  most  frequently  used  by  Safety.  Behavioral  analysis  is 
most  frequently  used  by  Transportation  (9)  and  Energy  and  Utilities  (9).  DIIM,  which  seems  to  be  mentioned  very  rarely 
with  a  specific  critical  infrastructure,  is  seen  most  frequently  within  Finance  (2). 
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4.3. 1.4  M&S  Techniques  by  Government  Level 

Another  way  to  explore  modeling  and  simulation  techniques  in  the  data  set  is  to  examine  who  is  using 
each  M&S.  Instead  of  analyzing  the  "who"  on  an  individual  basis,  a  fourth  thematic  group  called 
"Government'  was  created  to  group  together  the  main  user  groups  in  the  dataset.  The  Regional/local 
group  was  made  by  gathering  all  terms  with  the  words  "regional"  or  "local"  in  them.  The  top  terms 
include  local  government,  regional  planning,  regional  level,  local  level,  regional/local  infrastructure  etc. 
The  National  group  includes  national  infrastructure,  national  capital  region,  as  well  as  federal 
government  and  federal  agencies.  Government  includes  the  term  government,  government  agencies, 
government  services,  government  sector,  as  well  as  various  international  governments  as  in  U.S.,  U.K., 
Hong  Kong,  Chinese,  Dutch,  etc.  The  Department  of  Homeland  Security  is  created  by  grouping  together 
all  terms  that  refer  specifically  to  homeland  security  (e.g.  DHS,  policy,  agencies,  analysts,  modeling, 
operations,  directives  etc.).  Military/DoD  differs  only  slightly  from  the  Military  Cl  group  in  that  it 
includes  the  Defense  Industrial  Base  (which  is  found  in  the  Manufacturing  Cl  Sector).  International 
groups  together  various  terms  in  conjunction  with  the  word  'international'  (such  as  relations,  trade, 
level,  infrastructure  networks)  and  also  includes  the  United  Nations  and  terms  in  conjunction  with  the 
word  transnational. 

Figure  16  shows  the  co-occurrence  of  each  of  these  government  bodies  with  the  modeling  and 
simulation  techniques.  This  map  has  the  modeling  and  simulation  techniques  ordered  by  most  frequent 
co-occurrence  across  all  groups  (i.e.  Dynamic  M&S  is  used  most  across  all  6  government  bodies, 
followed  by  Estimation  etc.)  The  first  3-5  techniques  are  used  quite  regularly  by  each  government  body. 
Input-Output  modeling,  Uncertainty  analysis  and  Bayesian  techniques  are  used  most  frequently  by  the 
Regional/local  group,  whereas  Agent  based  modeling  is  used  most  by  the  National  group.  The 
Government  group  uses  Forecasting  more  than  any  other  M&S,  while  the  Military  uses  Game  Theory 
more  often  than  Dynamic  M&S. 


NRC 


Page  31  of  63 


Knowledge  Management 


CIPR  Literature  Survey:  Modeling  and  Simulation 


December  2013 


4.4  Interdependency  Topics 

Interdependency  modeling  of  critical  infrastructure  is  an  important  and  growing  field  of  study2,4.  Recent 
worldwide  events  such  as  2001  World  Trade  Center  attacks,  Hurricane  Katrina  and  Sandy,  the  2011 
earthquake  in  Japan  and  the  2013  typhoon  in  the  Philippines,  as  well  as  other  disasters,  have  shown  us 
that  interdependencies  between  critical  infrastructures  are  complex  and  can  result  in  cascading  failures. 
Interdependencies  may  be  in  the  form  of  physical,  cyber,  geographic  or  logical  dependencies.39 
Modeling  these  interdependencies  is  a  critical  step  in  identifying  the  real  vulnerabilities  of 
infrastructures  and  protecting  them.  Much  like  general  Cl  modeling  and  simulation  there  are  many 
different  approaches  as  will  be  reviewed  below.  Ouyang2  provides  a  good  overview  of  interdependency 
typologies  as  well  as  a  helpful  categorization  and  comparison  of  various  modeling  and  simulation 
approaches  in  the  Cl  interdependency  literature.  That  being  said,  there  are  multiple  ways  in  which  these 
models  are  related  and  there  is  no  single  taxonomy  or  classification  that  suits  all  purposes.40 

As  mentioned  earlier,  the  Interdependency  set  was  created  by  extracting  all  records  that  had  the  term 
interdependency,  independency,  dependent,  dependency,  regional,  or  local,  and  totalled  633.  All  figures 
and  tables  in  this  section,  unless  otherwise  specified,  are  based  strictly  on  the  633  record 
Interdependency  set.  All  of  the  analyses  that  were  performed  on  the  master  set  are  repeated  for  the 
interdependency  set  in  order  to  focus  in  on  the  specific  topic  of  modeling  and  simulating  critical 
infrastructure  interdependencies. 

As  described  above,  the  same  122  subject  groups  were  also  used  in  the  interdependency  set  to  group 
together  topics  based  on  terms  in  the  various  keyword  fields  and  from  words  in  the  title  and  abstracts. 
The  same  four  thematic  groups  of  Modeling  and  Simulation  Techniques,  Critical  Infrastructure,  Threats, 
and  Government  Levels  were  also  used  to  drill  down  into  the  data  to  answer  the  key  questions. 

A  subset  of  the  cluster  map  of  the  122  subject  groups  is  provided  in  Figure  17.  The  largest  red  cluster 
includes  many  of  the  Cl  groups  including  Safety,  Health  Care,  Government,  Military,  Cyber, 
Manufacturing  with  many  of  the  threats  e.g.  Disasters,  Terrorism,  Hurricane,  CBRNE, 
Disease/pan/epidemic  and  all  six  of  the  terms  in  the  Government  thematic  subject  group:  Regional/local, 
National,  Government,  Military/DoD,  Department  of  Homeland  Security  and  International.  More  general 
words  that  also  cluster  in  this  group  are  mitigation,  planning,  preparedness  and  SCADA. 

Next  there  is  a  large  yellow  cluster  in  the  center  of  the  map  that  groups  the  largest  critical  infrastructure 
groups  including  Energy  and  Utilities,  ICT,  Transportation,  and  Water  with  Finance,  and  Coastal  zones. 
This  cluster  also  includes  software  &  tools,  resilience,  recovery,  and  economic  issues.  The  inclusion  of 
software  &  tools  in  this  cluster  makes  sense  since  these  Cl  groups  are  the  predominant  users  of  the  tools 
found  in  the  dataset. 

As  in  the  master  set,  we  see  a  clustering  of  risk  assessment,  measurement  and  engineering  in  blue  next 
to  the  yellow  cluster.  These  are  again  connected  to  risk  analysis  which  is  clustered  with  RAMCAP  (Risk 
Analysis  and  Management  of  Critical  Asset  Protection)  in  brown,  qualitative  measures,  quantitative 
measures  and  quantitative  vulnerability  assessment  in  green,  Estimation  and  probability  in  orange, 
Uncertainty  and  Monte  Carlo  in  bright  green  as  well  as  Tsunamis  and  Earthquakes  in  pale  green. 
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Below  the  yellow  set  is  a  purple  cluster  bringing  together  detection,  anomaly,  intrusion,  sensors, 
monitoring  and  alerting  (early  warning).  While  above  the  yellow  set  is  a  blue  cluster  including  Floods, 
Climate  change  and  Extreme  Weather  which  correlates  with  the  adaptive  and  adaptation  cluster  in 
purple.  There  is  also  a  turquoise  cluster  just  to  the  right  of  this  one  that  brings  together  a  number  of 
modeling  techniques  including  Dynamic  M&S,  Input-Output  modeling,  DIIM,  and  System  Dynamics 
Modeling. 

Many  of  the  remaining  nodes  that  are  further  out  from  the  centre  (not  shown  here)  are  the  modeling 
and  simulation  techniques  as  well  as  threats  in  the  dataset.  While  the  M&S  are  not  all  clustering  with 
the  various  Cl  groups,  we  will  repeat  the  same  analyses  as  above  to  explore  the  relationships  between 
the  M&S  and  Cl,  M&S  and  threats  and  M&S  and  government  levels  in  section  4.4.1. 
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Figure  18  presents  the  number  of  records  by  year  for  each  critical  infrastructure  as  well  as  the  totals  for 
the  time  period  (2003-2014).  Since  the  interdependency  set  did  not  have  as  significant  a  drop  in  2012  as 
compared  to  the  master  set,  overall  we  do  not  see  as  much  of  a  drop  in  records  in  2012  for  the  various 
sectors.  Once  again  Energy  and  utilities  is  top  of  the  list.  This  is  unsurprising  since  virtually  all  industries 
are  reliant  on  power  and  fuel  and  communities  are  reliant  on  utilities,  all  the  other  sectors  have  some 
dependence  on  this  sector.8  Some  notable  drops  include  Government,  which  had  been  steadily  climbing 
since  2009,  but  dropped  by  more  than  half  (from  15  to  7)  in  2012.  Cyber  also  dropped  by  50%  between 
2011  and  2012,  and  Food  dropped  from  4  to  1  between  the  same  years.  On  the  other  hand  we  do  see 
some  growth  in  a  few  sectors  despite  the  drop  in  overall  records:  Safety  grows  from  17  to  20  in  2012 
and  Water  nearly  doubles  from  8  to  15.  Another  observation  that  can  be  made  by  comparing  the  two 
sets  is  that  Safety  and  Transportation  have  switched  place.  More  significantly,  there  are  relatively  more 
interdependency  modeling  and  simulations  publications  in  the  Water  sector  (now  in  5th  place  in  the 
interdependency  set  as  opposed  to  7th  in  the  master  set)  and  less  in  Cyber  (which  has  gone  from  5th 
place  in  the  master  set,  to  9th  in  the  interdependency  set).  This  rise  in  the  position  of  the  Water  sector 
may  be  due  to  the  great  number  of  interdependencies  that  exist  between  water  infrastructure  and 
many  of  the  other  Cl  sectors  including  Safety  (firefighters),  Health  Care,  Energy  and  Utilities, 
Transportation  and  Food.41 


Year 


Critical  Infrastructure 

Total 

2003 

2004 

2005 

2006 

2007 

2008 

2009 

2010 

2011 

2012 

Energy  and  Utilities 

250 

• 

4 

• 

12 

• 

17 

• 

8 

• 

36 

• 

19 

• 

38 

• 

33 

• 

34 

• 

ICT 

190 

2 

• 

11 

• 

12 

• 

17 

• 

21 

• 

19 

• 

30 

# 

29 

• 

23 

• 

Transportation 

168 

• 

2 

• 

10 

• 

12 

• 

10 

• 

21 

• 

6 

• 

25 

• 

29 

• 

24 

• 

Safety 

148 

• 

1 

• 

7 

• 

14 

• 

10 

• 

20 

• 

13 

• 

18 

• 

16 

• 

17 

• 

Water 

105 

• 

3 

• 

6 

• 

6 

• 

4 

• 

17 

• 

5 

• 

19 

• 

17 

• 

8 

• 

Health  Care 

79 

• 

2 

2 

• 

5 

• 

7 

• 

14 

• 

6 

• 

8 

• 

12 

• 

11 

• 

Government 

76 

• 

1 

• 

5 

• 

8 

• 

5 

• 

6 

• 

3 

• 

9 

• 

13 

• 

15 

• 

Military 

67 

2 

• 

3 

• 

7 

1 

• 

5 

• 

6 

• 

9 

• 

13 

• 

8 

• 

Cyber 

66 

• 

1 

• 

1 

• 

4 

• 

5 

• 

9 

• 

6 

• 

8 

• 

11 

• 

12 

• 

Finance 

53 

• 

2 

• 

5 

• 

4 

• 

3 

• 

8 

• 

13 

• 

5 

• 

6 

• 

1 

• 

Buildings 

49 

• 

1 

• 

5 

• 

5 

• 

9 

• 

6 

• 

7 

• 

9 

• 

Manufacturing 

40 

• 

4 

• 

2 

• 

3 

• 

6 

• 

3 

• 

4 

• 

9 

• 

5 

• 

Public  infrastructure 

29 

2 

2 

2 

3 

• 

3 

• 

3 

• 

4 

• 

3 

Food 

24 

• 

1 

• 

2 

• 

1 

• 

2 

• 

5 

• 

2 

• 

5 

• 

4 

• 

Coastal  zone 

22 

• 

3 

• 

4 

1 

• 

5 

• 

4 

2 

Figure  18.  Interdependency  set,  Critical  Infrastructure  by  Year,  2003-2012  (Totals  based  on  2003-2014.) 

The  same  threats  from  the  master  set  were  also  mapped  out  in  the  interdependency  set.  Figure  19 
shows  the  number  of  records  in  each  year  between  2003  and  2012  along  with  the  totals  for  2003-2014. 
Notable  changes  in  this  figure,  as  compared  to  the  master  set,  is  that  Disaster  interdependency 
modeling  and  simulation  now  surpasses  general  Terrorism  modeling  and  simulation  (first  in  the  master 
set  based  on  total  records  between  2003-2014).  Similarly,  if  we  look  just  at  the  number  of  records  in 
2012,  Terrorism  drops  from  2nd  in  the  master  set  to  3rd  in  the  interdependency  set,  whereas  Earthguake 
moves  from  4th  to  2nd.  CBRNE  has  also  dropped  from  4th  place  to  8th  based  on  total  records  between 
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2003  and  2014.  This  all  shows  that  there  are  slight  shifts  in  the  threat  areas  that  dominate  the 
interdependency  modeling  and  simulation  literature. 


Threat 

Total 

2003 

2004 

2005 

2006 

Year 

2007 

2008 

2009 

2010 

2011 

2012 

Disasters 

115 

• 

2 

• 

4 

• 

8 

• 

5 

018 

• 

9 

021 

• 

13 

• 

13 

• 

14 

Terrorism 

82 

• 

1 

• 

5 

• 

13 

• 

7 

0  11 

• 

6 

•  9 

• 

12 

• 

8 

• 

6 

Earthquake 

55 

• 

2 

• 

3 

• 

2 

3 

• 

2 

12 

• 

7 

• 

11 

• 

8 

Natural  Disaster 

45 

• 

1 

• 

2 

• 

4 

• 

2 

•  9 

• 

3 

•  6 

• 

3 

• 

4 

• 

6 

Floods 

33 

1 

• 

1 

• 

2 

5 

• 

3 

2 

• 

6 

• 

5 

• 

5 

Hurricane 

28 

• 

1 

• 

1 

•  7 

• 

2 

•  5 

• 

2 

• 

4 

• 

4 

CBRNE 

23 

1 

• 

3 

1 

2 

• 

3 

3 

• 

1 

• 

6 

• 

1 

Blackout 

24 

• 

1 

• 

1 

•  4 

• 

3 

•  6 

• 

1 

• 

5 

• 

2 

Fire 

23 

• 

3 

5 

• 

4 

1 

• 

3 

• 

3 

• 

4 

Extreme  weather 

19 

• 

3 

• 

2 

2 

•  4 

• 

3 

• 

2 

• 

3 

Climate  change 

19 

1 

2 

• 

4 

• 

4 

• 

5 

Disease/  pan/epidemic 

10 

• 

2 

2 

• 

1 

1 

• 

3 

Denial  of  service  Attack 

6 

1 

1 

• 

1 

• 

2 

Tsunami 

7 

1 

• 

3 

• 

1 

Drought  1  1 


Figure  19.  Interdependency  set,  Threat  by  Year,  2003-2012 

As  in  the  master  set,  a  Disasters  co-occurrence  map  was  made  to  see  which  specific  types  of  disasters 
are  most  frequently  studied  in  the  interdependency  set.  Figure  20  shows  that  the  majority  of  disasters 
are  virtually  the  same  as  in  the  master  set  with  Natural  disasters.  Terrorism,  Earthquake  and  Floods 
topping  the  list.  The  only  small  difference  is  that  Blackout  has  risen  to  roughly  the  midpoint  whereas  it 
was  much  closer  to  the  bottom  of  the  list  in  the  master  set.  Perhaps  this  is  due  to  the  fact  that  on  its 
own,  Blackouts  are  not  as  significant  a  threat  to  critical  infrastructure  as  it  is  when  you  take  into  account 
the  cascading  effects  it  has  on  other  sectors. 
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Figure  20.  Interdependency  set,  Co-occurrence  of  Disasters  with  Remaining  Threats 

4.4.1  Interdependency  Modeling  and  Simulation 

The  same  27  modeling  and  simulation  techniques  from  the  master  dataset  were  examined  in  the 
interdependency  set.  Table  5  presents  the  total  number  of  records  for  each  in  this  subset.  As  in  the 
master  set,  Dynamic  M&S  is  at  the  top  closely  followed  by  Estimation. 

Table  5.  Modeling  and  Simulation  Techniques,  Interdependency  set,  Number  of  Publications. 


M&S  Technique 

No. 

Publications 

M&S  Technique 

No. 

Publications 

Dynamic  M&S 

65 

Clustering 

10 

Estimation 

52 

Stochastic  modeling 

10 

GIS 

39 

System  dynamics  modeling 

10 

Input-Output  Modeling 

38 

Behavioral  Analysis 

9 

Classification/Pattern 

Identification 

36 

Monte  Carlo 

9 

Statistical/numerical 

techniques 

35 

Dynamic  Inoperability  Input- 
Output  Model  (DIIM) 

8 

Topological 

34 

Tree  Analysis 

8 

Graph  theory/model 

28 

Markov 

7 

Agent  based 

22 

Game  theory 

6 

Forecasting 

15 

Network  theory 

6 

Hierarchical  Methods  (other) 

15 

Petri  Nets 

6 

Uncertainty  Analysis 

15 

Hierarchical  Holographic 
Modeling 

4 

Bayesian 

14 

Quantitative  Vulnerability 
Assessment 

1 

Fuzzy 

13 

Significant  changes  between  the  master  and  interdependency  sets  based  on  the  positioning  of  the  M&S 
techniques  in  terms  of  their  number  of  publications  is  summarized  in  Table  6  below. 


Table  6.  Comparison  of  position  of  M&S  techniques  based  on  number  of  publications  between  the  master  and 

interdependency  sets. 


M&S  Technique 


Position  in 
Interdependency  Set 


Position  in  Master  Set 


Input-Output  Modeling 

4™  7\ 

10th 

Forecasting 

io  n 

r  Mth 

Bayesian 

13th 

18th 

DIIM 

20th 

28th 

Clustering 

15th 

22nd 

Game  Theory 

23rd  

12 

While  we  do  see  some  shifts  between  the  two  sets,  the  most  prominent  M&S  techniques  are  relatively 
similar.  The  top  five  in  the  master  set  are  Dynamic  M&S,  Estimation,  Statistical/numerical  techniques, 
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Classification/pattern  identification,  and  GIS  which  is  tied  for  5th  place  with  Graph  theory/model  whereas 
the  top  five  in  the  interdependency  set  are  Dynamic  M&S,  Estimation,  GIS,  Input-Output  Modeling  and 
Classification/pattern  identification.  The  only  notable  difference  in  the  top  five,  as  pointed  out  in  Table 
6,  is  the  rise  of  Input-Output  Modeling  into  the  top  5  in  the  interdependency  set.  Input-Output  Modeling 
is  based  on  Wassily  Leontief's  economic  input-output  model  and  conceives  of  interconnected  critical 
infrastructure  systems  as  having  inputs  of  failure  due  to  disasters  or  acts  of  terrorism  and  outputs  as  the 
risk  of  inoperability  due  to  the  inputs.42 

4.4. 1.1  Interdependency  M&S  Emerging  Trends 

In  order  to  identify  emerging  research  trends  in  the  modeling  and  simulation  of  interdependencies  in 
CIPR,  an  R&D  momentum  analysis  was  conducted.  Further  explanation  of  the  methodology  behind  the 
momentum  indicator  is  included  in  Section  7.3,  but  as  a  reminder,  this  technique  essentially  plots  the 
standard  deviation  of  standardized  measures  of  publication  counts  and  velocity  (the  rate  of  publication 
increase)  on  two  axes.  Nodes  which  plot  to  the  left  of  the  Y-axis  intersection  have  below-average 
velocity,  and  those  which  are  found  below  the  X-axis  have  relatively  smaller  publication  counts.  A  third 
dimension  is  added  by  sizing  nodes  relative  to  the  total  number  of  underlying  publications.  Even  a  small 
node  which  plots  to  the  right/lower  side  of  the  axes  may  be  of  interest,  since  emerging  topics  are 
typically  small  in  numbers  as  they  begin  to  attract  research  attention  and  increase  in  velocity.  Figure  21 
plots  all  27  modeling  and  simulation  techniques  based  on  the  records  in  the  interdependency  set. 
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Figure  21.  Interdependency  Set,  Modeling  and  Simulation  R&D  Momentum 


As  in  the  master  set,  the  majority  of  the  M&S  fall  below  the  x-axis.  M&S  that  are  already  "hot"  in  the 
field  with  high  number  of  records  and  a  high  rate  of  acceleration  closely  resemble  the  master  set  with 
Dynamic  M&S,  Estimation,  Statistical/numerical  techniques,  GIS,  and  uncertainty  analysis  repeating  in 
this  quadrant.  New  additions  to  the  top  right  quadrant  include  Agent  based  and  Topological  which  were 
established  trends  in  the  master  set.  The  only  established  trends  in  the  interdependency  subset  appears 
to  be  input-output  modeling  which  also  rose  in  the  M&S  ranking  in  Tables  5  and  6  .  In  order  to  assess 
whether  the  remaining  M&S  are  emerging  topics  in  relation  to  CIPR,  the  number  of  publications  for  each 
node  in  the  lower  right  quadrant  is  provided  for  the  past  5  yearsj. 


j 


Insufficient  data  prevented  us  from  presenting  a  figure  for  "new"  or  "disappearing"  techniques  in  the  interdependency  set. 
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Figure  22  presents  the  M&S  techniques  that  fall  in  the  lower  right  quadrant,  or  the  "emerging" 
techniques  quadrant.  Most  notably  we  can  see  a  significant  jump  in  the  use  of  Bayesian  techniques  and 
a  doubling  in  Tree  analysis  (as  was  seen  in  the  master  set  as  well).  Two  examples  of  the  use  of  Bayesian 
techniques  from  the  interdependency  dataset  include  Nechita,  Muraru  and  Talmaciu  who  used  a 
Bayesian  approach  to  uncertainty  to  create  a  framework  for  the  assessment  of  risk  probability  with  a 
particular  focus  on  information  systems  in  critical  infrastructure10  and  Di  Giorgio  and  Liberati  who 
presented  a  novel  approach  to  Cl  interdependency  analysis  based  on  Dynamic  Bayesian  Network  (DBN) 
formalism.13  Tree  analysis  has  been  used  to  study  the  impact  of  an  earthquake  on  the  interdependent 
critical  infrastructure  (including  power,  water  and  Transportation )  that  are  connected  to  nuclear  power 
plants,18  as  well  as  in  the  creation  of  tools  to  support  policy  makers  working  on  the  resilience  of 
interdependent  Cl  that  simulates  the  landscape  of  possible  outcomes  due  to  different  policies.43 
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Figure  22.  Interdependency  Set,  Modeling  and  Simulation  Techniques  by  Year,  Nodes  in  Lower  Right  Quadrant  - 

Emerging  Techniques. 
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Table  7  highlights  the  frequency  with  which  each  of  the  top  5  M&S  techniques  are  used  by  various 
threats  as  well  as  the  individual  threat  that  uses  each  technique  most  often. 

Table  7.  Frequency  of  use  of  top  five  interdependencies  M&S  by  various  threats 


M&S  Technique 

Frequency  of  use  by  threats 

Threat  that  is  the  most 
frequent  user  of  the  technique 

Dynamic  M&S 

Disasters  (14),  Natural  disasters  (11) 
Earthquakes  (10),  and  Terrorism  (9) 

Disasters 

Estimation 

Earthquakes  (10)  and  Climate  change  (8) 

Climate  change 

GIS 

Disasters  (9),  Floods  (7),  Terrorism  (6), 
Earthquake  (5)  and  Fire  (5) 

Fire 

Input-output  modeling 

Disasters  (12),  Terrorism  (10),  and 

Natural  disasters  (8) 

Terrorism 

Statistical/numerical  techniques 

Disasters  (9),  Earthquakes  (6),  CBRNE  (4) 
and  Floods  (4) 

CBRNE,  Disease/pan/epidemic 

Interestingly,  Dynamic  Input-Output  Inoperability  Model  (DIIM)  is  also  fairly  high  up  in  the  list,  and  is 
significantly  higher  than  in  was  in  the  master  set,  with  50%  of  these  records  dating  to  2013. k  Dynamic 
Input-Output  Inoperability  Model  (DIIM),  which  is  an  extension  to  the  static  Inoperability  Input-Output 
Model  (IIM)  is  based  on  Leontief's  Input-Output  model.  DIIM  is  designed  to  analyze  how  a  system  of 
interdependent  sectors  can  be  adversely  affected  as  a  result  of  initial  perturbations  to  other  sectors 
through  willful  attacks  or  natural  disasters.44  The  dynamic  extension  of  IIM  analyzes  different  temporal 
frames  of  recovery  and  characterizes  the  required  sector  adjustments  for  achieving  new  production 
levels.45  DIIM  is  used  most  frequently  in  Disasters  (7)  and  Natural  disasters  (5).  These  records  discuss, 
among  other  topics,  the  use  of  DIIM  along  with  the  Bureau  of  Economic  Analysis  (BEA)  and  the  Regional 
Input-Output  Multiplier  System  (RIMS  II)  commodity-flow  data  of  59  sectors  in  Virginia  to  model  a 
terrorist  attack  scenario46  as  well  as  in  a  denial  of  service  attack  on  IT  infrastructure  to  model  highly 
affected  economic  sectors,47  and  as  a  method  of  modeling  workforce  disruptions  in  the  aftermath  of 
hurricanes.48  Xu  and  his  colleagues  have  also  written  two  articles  in  2013  describing  a  variety  of  uses  of 
DIIM. 49,50  Unlike  in  the  master  set,  Game  Theory  is  rarely  used  in  interdependency  modeling  showing  up 
only  once  in  Terrorism  and  once  in  Denial  of  service  attacks. 

4.4. 1.3  Interdependency  M&S  and  Critical  Infrastructure 

Similar  to  exploring  the  overlaps  between  M&S  techniques  and  threats,  Figure  24  presents  a  co¬ 
occurrence  matrix  of  M&S  techniques  with  the  critical  infrastructure  presented  in  Figure  18.  The  most 
frequently  used  M&S  for  each  Cl  is  listed  in  Table  8. 


As  mentioned  previously,  data  from  2013  is  not  included  in  the  R&D  momentum  analyses  and  thus  may  explain  why  DIIM 
does  not  show  up  as  prominently  above. 
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Table  8.  Most  frequently  used  M&S  for  each  critical  infrastructure 


Critical  Infrastructure 

Modeling  and  Simulation  Technique 

Energy  and  Utilities 

Dynamic  M&S  (27) 

Transportation 

Dynamic  M&S  (17) 

ICT 

Dynamic  M&S  (16),  Topological  (16) 

Safety 

Dynamic  M&S  (15) 

Water 

Estimation  (13),  GIS  (13) 

Health  Care 

Estimation  (8) 

Finance 

Input-Output  Modeling  (9) 

Buildings 

Dynamic  M&S  (7) 

Government 

Dynamic  M&S  (5),  GIS  (5) 

Military 

Topological  (5) 

Cyber 

Dynamic  M&S  (5) 

Manufacturing 

Dynamic  M&S  (7) 

Coastal  zone 

GIS  (5) 

Public  Infrastructure 

Dynamic  M&S  (3),  Estimation  (3), 
Classification/Pattern  ID  (3) 

Food 

Dynamic  M&S  (4) 

Table  8  shows,  once  again,  that  Dynamic  M&S  is  the  most  frequently  used  modeling  and  simulation 
technique  in  the  interdependency  set,  with  a  total  of  65  records  and  10  Cl  sectors  favoring  it.  Six  other 
techniques  have  over  30  records  each  including,  Estimation  (52),  GIS  (39),  Input-Output  modeling  (38), 
Classification/pattern  identification  (36),  Statistical/numerical  techniques  (35)  and  Topological 
techniques  (34). 

The  majority  of  the  techniques  in  Figure  24  are  most  frequently  seen  in  use  by  the  Energy  and  Utilities 
sector,  however  this  is  possibly  due  to  the  fact  that  this  sector  is  quite  a  bit  larger  (in  terms  of  number  of 
records,  250)  than  the  others.  Similar  to  the  master  set,  we  see  a  predominant  use  of  Topological  M&S 
techniques  in  Energy  and  Utilities  (23)  as  well  as  ICT  (16),  (ICT  also  used  Dynamic  M&S  as  frequently  as 
Topological  techniques).  GIS  is  used  most  frequently  by  Energy  and  Utilities  (16)  followed  by  Safety  and 
Water  (13  each).  Input-Output  modeling  is  seen  most  frequently  in  Energy  and  Utilities  as  well  as 
Finance  (9  each).  Hierarchical  methods  are  used  most  frequently  by  ICT  (9).  Clustering  is  used  most 
often  in  ICT  (4),  Transportation  (4)  and  Water  (4).  Similarly  System  Dynamics  Modeling  is  most  often 
used  in  Transportation  (4),  Water  (4)  and  Energy  and  Utilities  (4).  Network  theory  is  most  frequently 
used  in  ICT  (4)  and  Energy  and  Utilities  (4).  We  only  see  Quantitative  Vulnerability  Assessment  in  Energy 
and  Utilities  (1). 
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4.4. 1.4  Interdependency  M&S  by  Government  Level 

Figure  25  displays  the  co-occurrence  of  each  type  of  government  body  with  the  27  M&S  techniques.  Because  the 
interdependency  set  was  created  by  extracting  all  records  from  the  master  set  that  included  the  terms 
"regional"  and  "local"  (in  addition  to  interdependency  and  it's  synonyms),  it  is  unsurprising  to  see  higher 
number  of  records  for  most  M&S  techniques  in  the  regional/local  group.  The  top  five  techniques  used  by  the 
Regional/local  group  include  Dynamic  M&S  (27),  Estimation  (24),  GIS  (22),  Statistical/numerical  techniques  (16) 
and  Input-Output  modeling  (15).  At  the  National  level,  Dynamic  M&S  (17)  is  followed  by  GIS  (11)  and  then  Agent 
based  techniques,  Input-Output  modeling,  and  Classification/pattern  identification  with  8  records  each. 
Government  uses  Dynamic  M&S  most  frequently  (5)  followed  equally  by  GIS,  Input-Output  Modeling 
Forecasting,  and  System  dynamics  modeling  (3  each).  The  Department  of  Homeland  Security  and  the 
Military/DoD  show  up  very  little  in  this  graph  with  only  one  record  in  a  few  M&S  techniques  except  for  Input- 
Output  modeling  which  has  2  records  in  the  Department  of  Homeland  Security  group.  This  is  somewhat 
surprising  since  we  know  that  the  DHS  is  working  on  CIPR  interdependency  but  this  may  be  due  to  their  work  in 
this  specific  area  not  being  captured  in  the  databases  that  were  searched  or  perhaps  to  their  work  being 
commissioned  by  associated  bodies  such  as  Sandia  National  Labs  or  Idaho  National  Labs  and  therefore  not  being 
grouped  as  a  DHS  record. 
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Figure  25.  Modeling  and  Simulation  Technique  Usage  by  Government  Levels,  Interdependency  set 
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Knowledge  Management 


4.4.2  Interdependency/Regional  Tools 

In  addition  to  the  groups  presented  in  the  master  set,  three  additional  groups  were  made1  to  capture 
Interdependency  tools  (with  a  total  of  135  records'”),  Regional  tools  (with  a  total  of  42  records)  and 
Government/business  continuity  tools  (with  a  total  of  14  records,).  Furthermore,  each  previously 
identified  tool  was  searched  for  in  the  master  set  and  a  group  was  created  with  76  individual  tools.  It 
should  be  noted  that  additional  tools  exist  but  are  not  searchable  due  to  the  lack  of  specificity  in  the 
searched  database  (i.e.  an  article  made  reference  to  a  "suite  of  tools"  but  did  not  name  them.  In  these 
cases,  some  other  useful  reference  to  the  tool  was  searched).  A  co-occurrence  matrix  of  the  M&S 
techniques  and  these  three  new  groups  was  created  and  is  presented  in  Figure  26. 
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Figure  26.  Modeling  and  Simulation  by  Interdependency  /  Regional  /  Gov't  Continuity  Tools 

Interdependency  tools  tend  to  use  Dynamic  M&S  (19)  nearly  twice  as  often  as  the  next  most  frequent 
modeling  techniques:  Agent  based  techniques  (10),  Estimation  (10),  Input-Output  modeling  (10)  and 
Statistical/numerical  techniques  (10).  DIIM  (8)  and  Petri  nets  (7)  are  relatively  high  on  the  list  as  well. 

Table  9  lists  some  of  the  tools  found  in  these  M&S  Techniques. 


1  These  three  groups  were  built  in  the  master  set  based  on  a  field  that  combines  abstracts  (in  full)  and  all  the  keyword  fields  that 
was  used  to  create  the  groups  presented  earlier  in  the  report. 

m  The  total  number  of  records  is  not  a  count  of  the  number  of  tools  in  the  group  but  the  number  of  records  (or  publications) 
that  discuss  one  of  the  tools.  For  instance,  there  are  13  records  that  discuss  RAMCAP. 


Page  47  of  63 


Table  9.  Top  M&S  Techniques  along  with  Selected  Interdependency  Tools 


M&S  Technique 

Selected  Tools 

Dynamic  M&S 

N-ABLE,  Rapid  Prototype,  Cl  Risk  Model,  and  CIPDSS 

Agent -based  Modeling 

N-ABLE,  IRRIIS,  CRIPS,  MIT  and  OMNet 

Estimation 

HAZUS-MH,  Cl  Risk  Model,  N-ABLE  and  IRRIIS 

Input  Output  Modeling 

RIPS,  ISA  and  REAcct 

Statistical/numerical  techniques 

Fragility  Curves,  Muir  Web,  and  PipelineNET 

DIIM 

DIIM 

Petri  Nets 

Fragility  Curves,  GeoPN,  Resilience  Assessment  Framework, 
Discrete  Event  Simulation 

The  Regional  tools  group,  on  the  other  hand,  uses  Dynamic  M&S  (13)  most  frequently  followed  by  Input 
Output  Modeling  (9),  DIIM  (8),  GIS  (5),  Forecasting  (5),  Estimation  (5),  Statistical/numerical  techniques 
(4),  and  Agent  based  techniques  (1).  Specific  GIS  tools  include  Rapid  Prototype,  Regression  Models, 
GeoPN,  GONE,  OMEGA,  SWMPT  while  specific  Forecasting  tools  include  Rapid  Prototype,  SERSCIS  ICT, 
MIMESIS  and  Secure  Mediation  Gateway  to  list  a  few. 

The  Government  and  business  continuity  tools  group  use  Dynamic  M&S  (2)  and  the  following  techniques 
one  time  each;  Estimation,  Agent  based  modeling,  Bayesian  techniques.  Stochastic  Modeling  and 
Uncertainty  Analysis.  The  two  tools  that  are  specifically  mentioned  by  name  in  this  group  include  the 
EASI  model  and  the  OTB  SAF  simulation  tool. 

Other  tools  that  are  specifically  named  and  grouped  in  the  Government  and  business  continuity  tools 
group  but  do  not  show  up  in  this  co-occurrence  map  (because  they  do  not  specify  a  technique)  include: 

•  Flowroute, 

•  Resilience  Evaluation  System, 

•  Infrastructure  Survey  Tool, 

•  Community  Preparedness  and  Response  (CPR)  model  for  local  governments, 

•  CERT  Resiliency  Engineering  Framework, 

•  RAMCAP-PLUS, 

•  EASI  model, 

•  OTB  SAF  simulation  tool, 

•  Infrastructure  Analysis  System  (IAS),  and 

•  OMEGA. 

A  second  co-occurrence  matrix  of  the  critical  infrastructure  and  these  three  groups  of  tools  is  presented 
in  Figure  27. 
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Figure  27.  Critical  Infrastructure  by  Interdependency  /  Regional  /  Gov't  Continuity  Tools 

Unsurprisingly,  we  see  that  the  five  largest  Cl  sectors  in  the  interdependency  set  are  the  ones  that  are 
most  frequently  using  interdependency  tools.  Table  10  lists  some  of  the  tools  in  the  top  sectors  along 
with  examples  of  scenarios  in  which  they  have  been  used. 
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Table  10.  Top  5  Sectors,  Tools  Used,  Scenarios  of  Use 


Sector  Tools  Scenarios  of  Use 

Energy  and  Utilities  MIT,  CIMS,  CIPDSS-PST,  RESC-MONITOR,  •  In  Canada,  the  M-CI2  has  been  used  to 

Flow  Assurance  by  Management  of  model  cyber  interdependencies  of  critical 

Uncertainties  and  Simulation  (FAMUS)  infrastructure  in  the  event  of  power 

and  MOCA-RP  software,  PSS®SINCAL,  outages51 

M-CI2 


ICT 


MIT,  CIMS,  RESCI-MONITOR,  M-CI2, 
OMNET++,  N-SMART 


•  OMNET++  has  been  used  to  simulate 
wireless  sensor  networks52 

•  Host-Based  System  Security  software 
agents  by  JTF-GNO  has  been  used  to 
assess  cyber  damage  to  command  and 
control  information  technologies,53 

•  (N-SMART)  has  been  used  to  assess  impact 
of  disasters  (e.g.  hurricanes)  on 
telecommunication  networks  amongst 
others54 


Transportation  N-ABLE,  CIMS,  CIPDSS  and  RESCI- 

MONITOR 


•  N-ABLE  has  been  used  to  simulate  the 
economic  impact  of  infrastructure 
interdependencies  due  to  shipping  port 
closures  in  the  Transportation  sector55 


Safety  CAX,  SNA-GIS,  N-SMART  tool,  CRIPS,  •  CAX  (computer  assisted  exercises)  has 

CIMS  and  PSS®SINCAL  been  used  to  simulate  Cl 

interdependencies  in  the  area  of  civil 
protection 

•  SNA-GIS  (a  social  network  analysis  - 
geographic  information  system  tool)  has 
been  used  to  analyze  Cl  hazards  and 
threats  to  security56 

Water  ADWICE,  PSS®SINCAL  •  ADWICE  for  real-time  anomaly  detection 

in  water  management  systems57 


One  tool  that  crosses  most  of  the  top  five  Cl  in  the  project  is  the  MICIE  platform.  MICIE  is  an  on-line  risk 
assessment  tool  that  can  be  used  in  scenarios  with  interdependent  Cl  including  power  distribution 
networks,  power  plants,  refineries,  water  distribution  networks,  transportation  systems  and 
telecommunication  networks.58,59 

Finance  is  in  the  7th  place  in  the  Interdependency  tools,  using  these  tools  more  than  would  be  expected 
given  that  they  are  the  10th  (out  of  15)  largest  Cl  in  the  interdependency  set.  The  Military  also  appear  to 
be  using  these  tools  less  than  expected,  given  that  they  are  the  8th  largest  Cl  in  the  interdependency  set 
but  place  15th  in  their  usage  of  these  tools. 

In  terms  of  the  Regional  tools,  Energy  and  Utilities  tops  the  list  with  13  records.  This  is  closely  followed 
by  Safety  (12),  Transportation  (10),  Government  (8)  and  Flealth  Care  (8)  -  both  of  which  are  below  the 
top  5  in  the  overall  interdependency  Cl  sector  rankings.  The  Government  Cl  sector  used  the  following 
tools: 

•  Complex  Event  Modeling, 

•  Simulation  and  Analysis  (CEMSA), 


Page  50  of  63 


•  the  Homeland  Security  Regional  Cooperation  Areas  (HSRCAs)  conceptual  model, 

•  Operational  Multiscale  Environment  Model  with  Grid  Adaptivity  (OMEGA)  to  model  the  release  of 
anthrax  in  the  national  capital  region,60 

•  RAMCAP,  and 

•  HAZUS-MH  to  name  a  few. 

The  Health  Care  sector  used  PipelineNET  to  model  safe  drinking  water61,  RAM-W,  OMEGA  and  RAMCAP 
repeatedly.  The  Military  and  Food  are  at  the  bottom  of  the  list  of  users  of  regional  tools. 

In  terms  of  Government/business  continuity  tools,  Energy  and  Utilities  and  Water  have  the  most  records 
(4)  each  followed  by  Government,  Transportation  and  Finance  with  (3)  each.  Flow  route,  Community 
Preparedness  and  Response  (CPR),  Infrastructure  Analysis  System  (IAS)  and  RAMCAP  Plus  are  discussed 
in  the  Energy  and  Utilities  and  Water  sectors.  RAMCAP  Plus  is  also  discussed  in  Safety  and 
Manufacturing.  The  CPR  model  is  discussed  in  the  Transportation,  Government  and  Finance  sectors  and 
the  CERT  Resiliency  Engineering  Framework  and  IAS  are  also  discussed  in  Government  and  Finance. 

Six  Canadian  affiliations  are  associated  with  various  interdependency  tools  or  techniques  including: 

•  University  of  British  Columbia,  Vancouver,  BC  -  development  of  interdependency  modeling  tools 
that  include  physical  and  social  interdependencies  to  aid  decision  support.62 

•  York  University,  Toronto,  ON  -  M-CI2  project  which  models  interdependencies  using  social  network 
and  reliability  modelling  techniques;  simulates  infrastructures  using  techniques  from  the  complex 
adaptive  systems  field51;  and  developed  the  GeoPetri  Net  system,  which  can  be  used  to  simulate  the 
complex  geographical  relationships  among  places  and  nodes.  A  case  study  involving  an  education 
layer  with  15  nodes  (schools)  and  a  transportation  layer  with  25  node  lines  (streets)  in  a 
geographical  information  system  is  presented.63 

•  University  of  Toronto,  ON  -  the  use  of  social  network  analysis  to  study  human  interactions  and  to 
analyze  interdependency  characteristics  of  critical  infrastructure  networks.64 

•  Concordia  University,  Montreal,  QC  -  an  integrated  modeling  method  for  simulating  the 
vulnerability  of  a  critical  infrastructure  for  a  hazard  and  the  subsequent  interdependencies  among 
the  interconnected  infrastructures.65 

•  University  of  New  Brunswick,  Fredericton,  NB  -  the  integration  of  context-aware  modeling  as  a  tool 
for  controlling  the  clustering  mechanism  through  which  the  eNetwork  self-organizes  its  services  to 
tune  its  resilience  according  to  the  dynamics  of  an  occurring  situation.66 

•  Ecole  Polytechnique  de  Montreal,  QC  -  a  concrete  set  of  tools  (consequence  curves  and  flexible 
cartographic  representations)  enabling  the  visualization  of  cascading  effects  and  management  of 
physical  interdependencies  among  the  Cl.67 


4.5  Lessons  Learned  in  Cl  Modeling  and  Simulation 

There  is  a  general  lack  of  specific  best  practices  or  lessons  learned  in  the  reviewed  literature.  This  may 
mean  that  the  field  of  modeling  and  simulating  critical  infrastructures,  and  in  particular  Cl 
interdependencies,  has  not  yet  matured.  In  general,  articles  that  did  provide  best  practices,  lessons 
learned,  or  recommendations  were  either  presented  at  a  higher  level  focused  on  managing  risks  or 
disaster  response  or  were  more  specific  to  the  tool  and  specific  case  study  being  presented  that  they 
were  not  easily  generalized  to  M&S  of  Cl  interdependencies.  Two  areas  that  did  seem  to  have  more 
general  recommendations  than  others  was  the  protection  of  Cl  from  cyber-attacks  and  climate  change 
impacts,  but  again  these  were  not  specific  to  M&S. 
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Ouyang2  identifies  four  key  areas  in  which  Cl  interdependency  M&S  research  challenges  lie.  These  areas 
are  reiterated,  to  various  extents,  by  other  authors  as  well.3,40,68'70 

•  Data  access  and  collection 

o  Difficult  to  access  data  or  lack  of  precise  data  is  a  key  problem  in  the  field. 

•  Comprehensive  modeling  and  analysis 

o  Due  to  interdependencies,  all  Cl,  including  those  that  receive  less  attention  such  as 
finance  and  commercial  and  government  facilities  should  be  included  in  Cl  modeling  to 
improve  disaster  mitigation  and  recovery. 

o  An  open  modeling  framework  to  capture  short  and  long  term  changes  and  evolution  in 
CIs  is  desired. 

•  Integration  and  co-simulation 

o  The  variety  of  tools  and  techniques  for  M&S  capture  different  aspects  of 

interdependencies  with  varying  levels  of  error  and  uncertainty  and  potentially 
conflicting  results. 

o  Integration  of  these  techniques,  with  clear  roles,  into  a  uniform  framework  is  needed. 

•  Validation  and  applications 

o  Validating  new  models  outputs  to  historical  data  (the  norm)  is  not  always  appropriate  as 
Cl  and  their  environments  are  constantly  changing, 
o  A  comprehensive  and  standard  set  of  metrics,  guidelines  and  standards  are  needed  to 
demonstrate  the  applicability  of  different  modeling  and  simulation  approaches  and  how 
they  can  inform  different  types  of  decisions. 

In  2011,  the  U.S.  Department  of  Homeland  Security  and  the  National  Institute  of  Standards  and 
Technology  produced  a  series  of  recommendations  on  M&S  Cl  systems  (not  specific  to 
interdependencies)  for  homeland  security  applications.70  They  identified  several  issues  that  remained  to 
be  addressed  in  the  field,  including  but  not  limited  to: 

•  Identification  of  appropriate  models,  simulations,  tools  and  databases  to  address  critical 
infrastructure  and  key  resources  (CIKR)  analysis  needs,  that  can  be  shared  by  sector  specific 
agencies  with  the  user  community; 

•  Identification  of  technical  gaps  for  models,  simulations,  tools  and  databases; 

•  Development  of  system  requirement  specifications  for  CIKR  models,  simulations,  tools  and 
databases; 

•  Development  of  systems  dynamics  models  for  addressing  strategic  issues  for  different  CIKR 
sectors  and  systems; 

•  Development  of  mechanisms  to  enable  access  to  and  usage  of  CIKR  M&S  applications  by  Sector 
Specific  Agencies,  CIKR  partners,  and  operational  personnel; 

•  Development  of  simulation  application  architectures  to  enable  module  integration  and  standard 
data  interfaces  to  import  data  from  external  databases; 

•  Use  of  a  system  of  systems  engineering  approach  to  the  development  of  applications; 

•  Development  of  M&S  applications  as  open  systems; 

•  Use  of  object-oriented  models  in  CIKR  M&S; 

•  Integration  of  CIKR  models  and  simulations; 

•  Establishment  of  security  and  protection  mechanisms  for  sensitive  data; 
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•  Ownership  and  usage  of  publicly  vs.  privately  developed  models,  simulations,  tools,  and 
databases;  and 

•  Return  on  investment  to  stakeholders  and  sponsors  for  research  projects. 

Both  of  these  overviews  of  recommendations  highlight  the  lack  of  maturity  in  the  field  and  the  need  for 
continued  development. 


5  CONCLUSIONS 


To  help  support  CSS  in  their  long  term  goal  of  developing  an  integrated  national  and  regional  Critical 
Infrastructure  (Cl)  Dependency  model  for  Cl  risk  analysis  and  risk  mitigation  in  support  of  the  Cl  strategy, 
NRC  Knowledge  Management  has  performed  an  exploratory  study  of  existing  scientific,  industrial  and 
government  (domestic/international)  literature  on  critical  infrastructure  protection  and  resilience  (CIPR) 
and  has  reported  on  modeling  and  simulation  tools  and  trends. 

The  number  of  different  tools  that  can  be  used,  are  being  designed  or  researched  to  model  and  simulate 
CIPR  is  quite  high.  There  were  over  600  records  that  mention  tools  or  software  in  the  master  set  of 
2151.  While  these  tools  were  not  individually  listed,  113  tools  used  for  interdependency  modeling  or 
regional/local  modeling  or  government/business  continuity  were  examined.  The  majority  of  these  tools 
(82)  are  grouped  as  Interdependency  modeling,  with  18  falling  into  the  Regional  group  and  13  in  the 
Government/business  continuity  group. 

The  results  of  this  study  show  that  there  are  not  many  differences  in  modeling  and  simulation 
techniques  that  are  used  for  general  CIPR  or  CIPR  interdependencies.  Dynamic  Modeling  and  Simulation 
is  the  most  frequently  used  technique  for  CIPR  both  in  the  tools  that  were  examined  and  from  a 
theoretical  standpoint  in  the  data  sets.  Dynamic  M&S  can  be  found  in  N-ABLE,  Rapid  Prototype,  Cl  Risk 
Model,  and  CIPDSS  to  name  just  a  few  specific  tools.  Estimation,  Statistical/Numerical  techniques, 
Classification/Pattern  Identification,  GIS,  Graph  Theory/Model,  and  Input-Output  modeling  were  among 
the  top  techniques  as  well. 

In  terms  of  CIPR  M&S  in  general,  the  use  of  Bayesian  techniques  and  Fuzzy  techniques  can  be  seen  in 
the  "brand  new"  quadrant  of  the  R&D  momentum  analysis.  In  terms  of  emerging  techniques,  Tree 
Analysis  as  well  as  Behavioral  Analysis  and  Monte  Carlo  techniques  are  gaining  popularity  for  general 
CIPR  M&S.  In  terms  of  emerging  techniques,  Bayesian,  Tree  Analysis  and  Monte  Carlo  are  proving  to  be 
useful  for  interdependency  M&S. 

CIPR  modeling  and  simulation  as  well  as  interdependency  tools  are  most  frequently  used  in  the  Energy 
and  Utilities  sector  followed  by  ICT,  Transportation  and  Safety  both  in  terms  of  individual  and 
interdependency  modeling.  These  tools  are  most  frequently  applied  to  model  or  simulate  CIPR  in 
response  to  Disasters,  Terrorism,  Earthquakes  or  Natural  Disasters. 

The  United  States  followed  by  Italy  and  Canada  are  the  predominate  nations  working  on  CIPR  modeling 
and  simulation  as  per  our  dataset.  Within  the  United  States,  the  National  Laboratories,  including  Sandia 
and  Idaho  have  extensive  CIPR  programs  and  are  producing  interesting  tools  and  technologies.  In  Italy, 
the  main  author  affiliations  are  also  governmental  including  ISPRA  and  ENEA,  both  of  which  are  working 
on  Pan-European  CIPR  projects.  Within  Canada,  the  University  of  British  Columbia,  York  University  and 
Western  University  figure  prominently,  all  of  which  are  involved  in  either  the  Joint  Infrastructure 
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Interdependencies  Research  Program  or  the  Disaster  Response  Network  Enabled  Platform.  A  few 
additional  affiliations  rose  to  the  top  of  the  list  in  the  interdependency  set  including  Polytechnic 
University  of  Milan  (Italy),  Huazhong  University  of  Science  and  Technology  (China),  University  Campus 
Bio-Medico  of  Rome,  Rice  University  (U.S.),  the  Italian  National  Research  Council  (CNR),  TNO  Defense 
(Netherlands)  and  Gjpvik  University  College  in  Norway. 

Finally,  the  similarities  between  the  two  data  sets  (Cl  M&S  and  Cl  interdependency  M&S)  are  quite 
notable.  It  is  possible  that  the  subfield  of  CIPR  interdependency  modeling  and  simulation  is  still 
depending  on  tools  and  techniques  used  in  general  CIPR  modeling  because  it  has  not  yet  developed 
enough  of  its  own  tools  and  techniques.  The  only  exception  is  the  advancement  of  Input-Output 
Modeling  to  include  the  Dynamic  component  in  the  creation  of  Dynamic  Inoperability  Input-Output 
Modeling  to  specifically  model  interdependencies.  Despite  this  modest  advancement,  Ouyang2  argues 
that  there  is  still  a  need  for  a  new  concept  that  can  "integrate  different  approaches  into  a  single 
framework  and  co-simulation  platform  to  address  different  aspects  of  interdependent"  critical 
infrastructure.  This  alternative  view  is  supported  by  the  general  lack  of  specific  best  practices  or  lessons 
learned  in  the  literature  that  was  reviewed,  i.e.  the  field  of  modeling  and  simulating  critical 
infrastructures,  and  in  particular  Cl  interdependencies,  may  not  yet  have  matured  enough  to  have  best 
practices.  In  general,  articles  that  did  provide  best  practices,  lessons  learned,  or  recommendations  were 
either  presented  at  a  higher  level  focusing  on  managing  risks  or  disaster  response  or  were  more  specific 
to  the  tool  being  presented  and  not  easily  generalized  to  M&S  of  Cl  interdependencies.  Alternatively  it 
may  be  that  the  tools  and  techniques  that  are  used  in  general  CIPR  modeling  are  sufficiently  applicable 
to  interdependency  modeling.  More  time  and  testing  of  these  techniques  would  be  required  to 
determine  the  true  cause  of  the  overlap.  Revisiting  the  numerous  projects  and  programs  that  were 
presented  in  the  key  players  section  would  allow  one  to  keep  a  finger  on  the  pulse  of  developments  in 
this  area. 
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7  METHODOLOGY 


7.1  Search  Strategy 

A  literature  search  was  conducted  in  the  Scopus,  IE  Compendex,  Inspec  and  National  Technical 
Information  Service  (NTIS)  databases.  The  basic  search  strategy  used  to  retrieve  records  was  based  on 
keywords  shown  in  Table  11  below.  Terms  or  phrases  in  columns  A  and  B  were  combined  using  Boolean 
and  proximity  operators  (AND,  OR,  NEAR)  to  cover  all  aspects  of  the  problem.  The  search  targeted 
substantive  fields  such  as  title,  keywords  (controlled  and  uncontrolled  vocabularies)  and  abstracts.  The 
time  period  was  limited  to  records  2003  to  the  present. 


Table  11.  Search  Terms 


A:  Critical 

B:  Protection 

C:  Modeling  and 

D:  Risk/threats 

E:  Interdependency 

infrastructure 

/resilience 

simulation 

•  Critical 

•  Protection 

•  Model* 

• 

risk 

•  Interdependen* 

infrastructure 

•  Resilien* 

•  Simulation* 

• 

threat 

•  Regional 

•  Critical  assets 

•  Simulate 

• 

vulnerability* 

dependenc* 

•  Public 

•  Tool 

• 

disaster* 

•  regional 

infrastructure 

•  Software 

• 

Emergenc* 

interdependenc* 

•  Assess 

• 

Earthquake* 

• 

Tsunami* 

• 

All  hazard*  or  multi 

hazard*  or  hazard* 

• 

(extreme  or 
unforeseen)  event* 

• 

Flood*  or  drought  or 

fire*  or  wildfire* 

• 

Terroris*  or  extremis* 

• 

Epidemic*  or 
pandemic*  or  health 
NEAR  emergenc* 

• 

CBRN*  or  ((chemical  or 
biolog*  or  radioactive 
or  nuclear)  NEAR 
warfare  or  weapon*  or 
attack*)) 

• 

(Industrial  or 
Transportation  or 
chemical  )  NEAR 
(accident*  or  spill*  or 
leak*) 

• 

cyber  attack  or 
cyberattack 

• 

cyber  warfare 
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7.2  Analytic  Tools 

All  references  were  downloaded  into  VantagePoint  software  for  analysis.  VantagePoint  enables  the 
creation  of  various  groupings,  statistical  analyses,  matrices,  graphs,  and  cross-correlations  to  analyze  the 
data  and  profile  the  activities  of  the  major  players.  Once  in  VantagePoint,  duplicate  records  in  the  data 
were  removed  and  the  records  were  prepared  for  analysis. 

Keywords,  identifiers  (usually  author-supplied  keywords),  descriptors,  subject  headings  and  terms  from 
titles  were  merged  together  to  facilitate  subject  analysis.  These  terms  were  then  cleaned  to  harmonize 
variant  spellings,  acronyms  and  similar  meanings.  They  were  then  organized  into  subject  groups,  which 
were  defined  based  on  the  top  terms  and  readings  from  references  included  in  the  dataset.  Groups 
created  during  the  analysis  are  provided  as  an  attachment  to  this  record,  file  name:  KM-NRC  18526-122 
Subject  Groups.  The  four  Thematic  Groups  are  provided  in  section  7.2.1,  Table  12. 

Different  analytical  tools  were  used  to  generate  graphs  based  on  statistical  operations  performed  in 
VantagePoint.  TouchGraph  software  was  used  for  cluster  analysis  and  visualization  of  the  subject  groups 
while  Tableau  software  was  used  to  generate  bubble  graphs  and  some  tables. 


7.2.1  Thematic  Groups 

Table  12.  Thematic  Group 


Thematic  Group 

Scope 

Subject  Groups 

Modeling  and 

The  modeling  and 

Dynamic  M&S 

Fuzzy 

Simulation 

simulation  techniques 

Estimation 

Behavioral  Analysis 

Techniques 

used  in  Cl  PR 

Statistical/numerical 

techniques 

Classification/Pattern  ID 
GIS 

Graph  theory/model 
Topological 

Agent  based 

Input-Output  Modeling 
Hierarchical  Methods 
(other) 

Game  theory 

Uncertainty  Analysis 
Forecasting 

Stochastic  modeling 

Bayesian 

Monte  Carlo 

System  dynamics  modeling 

Tree  Analysis 

Clustering 

Petri  Nets 

Markov 

Network  theory 

Dynamic  Inoperability  Input-Output 
Model  (DIIM) 

Hierarchical  Holographic  Modeling 
Quantitative  Vulnerability 

Assessment 

Critical 

Physical  structures  or 

Energy  and  Utilities 

Buildings 

Infrastructure 

systems  impacted  by 

ICT 

Military 

disasters.  10  are  based 

Safety 

Finance 

on  the  Canadian  Cl 

Transportation 

Manufacturing 

sectors.  Additional  ones 

Cyber 

Public  Infrastructure 

are  added  due  to  their 

Government 

Coastal  Zone 

prevalence  in  the  dataset 

Water 

Health  Care 

Food 

Threats 

Threats  or  accidents 

Terrorism 

Climate  change 

which  may  lead  to 

Disaster 

Extreme  weather 

emergencies  or  to 

Earthquake 

Disease/pan/epidemic 

Page  61  of  63 


Thematic  Group 

Scope 

Subject  Groups 

disasters 

CBRNE 

Natural  Disaster 

Flood 

Fire 

Hurricane 

Blackout 

Denial  of  service  attack 

Tsunami 

Drought 

Government 

Various  levels  of 

Regional/Local 

Military/DoD 

Levels 

government  or  military 
that  figure  prominently  in 
the  dataset 

National 

Government 

Department  of  Homeland 
Security 

International 

7.3  R&D  Momentum 

To  ascertain  the  normalized  growth  rates  and  compare  values  according  to  their  standard  deviation  for 
each  of  the  subject  groups  in  the  thematic  groups,  we  plotted  publication  numbers  and  the  angle  (slope) 
of  their  increase  or  decline  over  time  over  time  (2008-2012),  using  linear  regression.  Average  slope 
degrees  and  standard  deviation  were  then  calculated  and  standardized,  to  produce  Z-scores,  a  measure 
of  the  relative  rate  of  publishing  increase  (velocity). 

Standardized  publication  counts  were  also  produced  for  each  of  the  subject  groups  analyzed.  Plotted  on 
2-dimensional  intersecting  axes,  the  z-scores  (velocity)  and  standardized  publication  counts  (mass) 
provide  an  expression  of  the  relative  momentum  of  topics  within  each  group. 

This  indicator  is  designed  to  identify  rapidly  rising  subjects  with  relatively  few  publications.  The 
challenge  of  identifying  such  subjects  lies  with  the  publication  volume  as  a  confounding  factor,  for  their 
rapid  growth  and  evolution  is  dwarfed  by  the  high  volume  of  established  subjects.  Specifically,  the 
notion  of  "emergence"  consists  not  only  of  a  sharply  rising  trend  line  but  also  of  a  small  footprint  (low 
publication  numbers)  in  the  domain  of  interest.  A  relatively  small  footprint  is  the  reason  emerging 
subjects  are  often  overlooked  until  their  disruptive  impacts  become  obvious. 

The  four-quadrant  visualization  provides  a  structured  view  of  the  relative  position  of  these  subjects 
within  the  group.  The  indicator  can  also  be  applied  to  identify  differences  between  topics  with  small 
publication  numbers  but  relatively  greater  velocity. 

A  third  dimension  is  added  by  relating  the  size  of  each  node  to  its  actual  publication  count. 
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Established  topics:  higher 
numbers  but  lower  rate  of 
acceleration 


"Hot"  topics:  higher  numbers, 
higher  rates  of  acceleration 
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Figure  28.  4  Quadrants  Momentum  Indicator 

One  possible  limitation  of  the  methodology  may  be  absent  or  incomplete  publication  values  for  certain 
years  in  a  time  series,  so  the  analysis  provided  by  z-scores  should  also  be  accompanied  by  a  review  of 
actual  values  over  time  wherever  possible. 


7.4  Sources  Consulted 

Licensed  databases: 

•  Scopus 

•  El  Compendex 

•  Inspec 

•  National  Technical  Information  Service  (NTIS) 
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